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INTRODUCTICai 

Stage Two of the Title IV Quality Control Project represents the 
first integrated study of five related student financial aid programs. 
Previous projects have focused on these programs separetely, in keeping 
with the structural differences among them. However^ there exist many 
similarities between the delivery systems for these programs which 
justify the current approach to quality control. Results of the previous 
studies/ taken together with regulatory changes and recent Department of 
Education (ED) policy initiatives, have led to an increased a'-.areness of 
the nature and patterns of error that exist in the administration of 
these programs, as well as the likely impact of corrective actions upon 
the level of error. 

In this introductory chapter we will present the major objectives of 
Stage Two, and discuss them in the context of the previous quality 
control studies that have shaped the current effort. We will also 
briefly review the Title IV programs themselves and introduce the data 
sources we used to conduct the study. Finally, we will describe how ;:his 
volume is related to the other reports that document this study. 

Subsequent chapters will describe in detail how the study was 
conducted. In Chapter 2 we will discuss how we have defined error in 
Stage Two and our procedures for measuring error* Chapter 3 will present 
the sampling methodology that we used to construct a national sample of 



Federal student aid recipients. Chapters 4 and 5 will describe our data 
collection activities/ and Chapter 6 will focus oa the preparation of ths 
data for analysis. Finally/ Chapter 7 will present a discussion of 
nonresponse and variance estimates and their effects on the sample. 

1*1 FCCDS OF STAGE TdO 

The Pell Grant Quality Control Study and Stage One of the Title IV 
Quality Control Project sought to identify and measure error in the Pell 
Grant program. Also, in Stage One, in the Campus-Based and Guaranteed 
Shudent Loan (GSL) programs/ the Title IV Quality Control Project sought 
to attribute error to its sources and to develop appropriate corrective 
actions to reduce error. The objectives of previous QC studies have been 
to measure error and create an awareness that error exists in the Title 
IV programs/ to support such corrective actions as the validation of 
income-related data, and to increase awareness of the need for quality 
control at the institutional level. A specific objective of Stage One of 
the Title IV Quality Control Project was to develop a methodology for 
measuring error in the Campus-Based programs and GSL certification 
process. These objectives have largely been achieved. Error in the five 
Title IV programs can be defined and measured. 

Stage Two represents a change in focus from the previous quality 
control (QC) studies in several ways. This study uill again measure 
error in the Title IV programs, but will go beyond the identification and 
measurement of specific errors and types of errors. Rather, our focus in 
stage Two is on ED's major policy initiatives - institutional quality 
control > intensified institutional verification/ and simplification. 
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Major Objectives of Stage Two 



The current study has five major objectives: 

♦ To c^etermine whether error persists in the Title IV programs 

♦ To establish whether patterns of error exist that indicate 
problems across the Title IV programs 

♦ To show the effects of prior ED corrective action initiatives 

♦ To measure residual error 

♦ To describe the effects of proposed major corrective actions 
on improving quality in the delivery of Federal student aid. 

The current level of awareness of error and its sources / as well as 
the recognition of the need for institutional quality control, have led 
to new policy initiatives on the part of ED and therefore a need for a 
different focus in Stage Two. Factors which have influenced the ED 
policy environment since the QC studies were initiated are shown in 
Exhibit 1-1. 



• Increasing budgetary pressure in the Title IV programs 

• IncreasKid understanding of the nature of error in the 
Title IV programs 

• Realization that a growing proportion of error is resistant to 
marginal/incremental change 

• Awareness of fundamental, structural problems in 
student aid programs 

• Increasing awareness of burden and limits of mandated 
verification as sole approach to removing error 

EXHIBIT 1-1 

FACTORS INFLUENCING CURRENT ED POLICY ENVIRONMENT 



ED has undertaiken three major policy initiatives in response to these 
factors: validation, institutional quality control/ and simplification 
(Exhibit 3-2). 
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tXHIBIT 1*2 

DEPARTMENT OP IDUCATION POUCY INTTIATIVES 



Validation in the Pall Grant program has reduced many application 
errors. While mandated by Federal regulation^ validation is essentially 
institutionally-oriented/ requiring verification of specified application 
items by the institution. It has boen particularly effective in reducing 
income-related application error and has been extended to the 
Campus-Based and GSL programs for 1986-87. 



In the area of institutional quality control. ED has accepted the 
findings of earlier studies which have shown that the presence of 
institutional quality control procedures is associated with lower levels 
of error. The Institutional Quality Control Pilot Project, begun in 
1985/ establishes a formal, cooperative partnership between ED and the 
participating pilot institutions to develop these types of procedures. 




These institutions assess their own policies and procedures/ measure 
etr>rs. and in-plem6nt specific activities to reduce error. '^Ae pilot 
project b^.o confirmed both the validity of this approach and its 
potential for error reduction. 

In t>ie third policy area, simplification/ ED has examined ways ia 
which the formulae for de^ermining a student's award or need for 
financial aid can be simplified to reduce error by using fewer data items 
which can be reliably verified/ while preserving the ability of the 
formula to sensitively measure need. Previous efforts in this area were 
focused on the Pell Grant program/ and demonstrated the efficacy of using 
a simplified formula. 

Stage Two also represents a change in focus from previous .studies irx 
that those studies were designed to measure error/ first in the Pell 
Grant program alone/ and then in Campus-Based and GSL certifications, 
apart from Pell. In each of these studies the sample of students was 
drawn to represent the population of recipients in the respective 
progr-^ms. In Stage Two, our study design called for a nationally 
representative sample of Title IV recipients. We can therefore assess 
the patterns of error that exist across the programs/ and develop 
corrective action recommendations to address these errors. The 
distribution of Pell, Campus-Based/ and GSL recipients in the sample has 
still allowed us to measure error by program/ as is necessary because 
differences in program structure not only preclude the development of a 
single error measure/ common to all five Title IV programs/ but also 
require different error measures for each program. We have also 
preserved different error definitions from prior studies in order to 
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assure comparability with them, so that trends in error and residual 
error can be analyzed* 

1.2 RELATI(»SHIP TO PREVIOUS QC STUDIES 



Over the past 10 years, a series of quality control studies has 
demonstrated that error can be identified and measured, and that analysis 
of error can result in the identification and implementation of 
corrective actions which can reduce error. These studies are shown in 
Exhibit 1-3. 

Each of these studies has built upon the findinys of its predecessors 
to refine its own methodology and to lead to increased understanding of 
the nature and causes of error in the Title IV programs. Study 
methodology has been sufficiently refined through these studies to 
provide precise estimates of the magnitude of aggregate error. 
Successive studies have pointed to the need for corrective actions to 
reduce the level of error in the Title IV programs without compromising 
program intent/ based on careful analysis of levels and sources of error, 
as well as importance of specific errors in relation to program 
objectives. 



1.2.1 Sinilaritiea to Previous Studies 



Stage Two is one of a series of quality control projects and, in that 
context, the ability to compare the results of this study with prior 
findings is vital to the success of both the project itself and ED's 
quality control initiatives. The methodology for Stage Two mu-'t provide 
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1984 • 85 




1986 



Office of Edtx:ation study, ccmparad Internal Revanue Saivice 
records with applicant data 



Student Rnandal Assistance Study Group report, based on 
public testinrK)ny, previous studies, and audits 



Basic Education Opportunity Grant (QEOQ) Study, examined 
application data ana institutional records 



Peti Grant Quality Control Study, consisting of two large national 
sun^s, studied enror in the Pell Grant Program, compared 
delivery systems* assessed options for redesigning delivery 
systenris, developed tha Inatttuttenal QuelHv Contml Handbook 



Titia IV Quality Control Study, Stage One, u national sun^ey of recipients 
of Canpus-Based aid and Guaranteed Student Loan certifications, studied 
error in the Campus*Based aid programs and GSL cerUfications 

THIe IV QuaWy Control Study, Stage Two, a national suwey of recipients 
of Pel! Grants, Campua-Based aid, and GSL certifications, studied 
error h the Title IV progranw 



The Institutional Quality Control Pilot Project, demonstrated the 
appiicabiDty of quality control to financial aid at the Institutional level 



• Guaranteed Student Loan QuaRty Control Project, 
identified and measured error in the GSL program from 
financial institutions, and guarantee agencies, and ED 



EXHIBIT 1-3 
QUALITY CONTROL STUDIES 
1975 • 1986 



the basis for analyses which will provide ED policy makers with trend 
data on error in order to evaluate the effectiveness of prior corrective 
actions and program changes. In order to accommodate this need, we have 
structured the measurement and analysis of error in conformance with 
prior studies. For example, we have maintained the basic definitions and 
structure for decomposition of errors. 

The design of Stage Two features many of the same elements as the 
Stage One and Pell QC studies. As shown in Exhibit 1-4/ the research 
methodology is essentially the same, having the following basic 
components : 

• Specification of the delivery system and its error points 

• Identification of the types and sources of available data 

• Definition of program error 

• Identification of potential causes and their relationship to error 

• Identification of corrective actions and analyses of likely 
benefits and costs 

• Determination, collection, and processing of the required data 

• Analysis and reporting of results. 

The presentation of study findings from Stage Two will also share 
elements of the previous studies. Findings will consist of both 
aggregate error measures and decomposed error measures to show the 
attribution of program-wide error to 'ts sources. The findings will also 
form the basis for corrective actions analyses. 
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EXHIBIT 1-4 

METHODOLOGY FOR QUALITY CONTROL STUDIES 



1.2,2 Review of Title IV Programs ^ 

The Title IV programs are so aamed because they are funded under 
Title IV of the Higher Education Act of 1965, as amended. This study is 
concerned with the Pell Grant program, the Campus-Based programs 
National Direct Student Loans (NDSL), Supplemental Education Opportunity 
Grants (SEOC;, and College Work-Study (CW-S), .nd the Guaranteed Student 
Loan (GSL) program. Major characteristics of the Title IV programs are 
shown in Exhibit 1-5. Another Title IV program. State Student Incentive 
Grants (SSIG), under which Federal funding is provided to states for the 
purpose of stud£nt financial assistance, is not included. 
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Pell Grantii are entitlements which support part of the cost of 
pursuing postsecondary education, up to specified limits. The students, 
the schools they attend, and the programs in which they enroll must 
satisfy categorical eligibility requirements. Once these requirements 
have been met, the amount of the student's grant depends on the student's 
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financial eligibility, which is a function of both the cost of the 
educational program and the student's ability to pay that cost from the 
student and family assets and income. The calculation of costs, the 
student/ family contribution. and awards are prescribed by program 
formulae. The money for awards is usually allocated from the U.S. 
Treasury to the institutions, which pay it to individual students or 
credit it to their accounts. 

The Cawpus-Based programs consist of three individual Federal 
student aid programs, each administered by participating postsecondary 
institutions: the National Direct Student Loan (NDSL) . renamed the 
Perkins Loan after reauthorization of the Higher Education Act. the 
College Work-Study (CW-S). and the Supplemental Educational Opportunity 
Grant (SEOG) programs. Individual institutions are free, within the 
regulations, to establish the parameters within which Campus-Based aid is 
awarded. Financial aid administrators at these institutions award 
Campus-Based funds, in conjunction with other programs, to meet student 
need as determined by a need analysis procedure approved by the 
Department of Education. Most often software is used to compute need 
according to the Uniform Methodology or summary need calculations 
supplied by service organizations. Campus aid administrators tailor 
awards to meet this need according to available funds, the institutional 
aid packaging policy, and any circumstances unique to the student. This 
aid packaging policy may dictate the sequence, amount, or types of aid 
given and the percentage of need met for different types of students. 
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Operationally, the administration of these three programs is 
significantly different from other Federal student aid programs. A 
financial aid officer at any of the participating institutions: 



♦ Could choose in the I'^aS-se program year from among a number 
of approved need analysis methodologies to calculate expected 
family contribution 

♦ Could deviate from the expected family contribution in 
individual cases provided the adjustments are documented 

♦ Could use one of sevsral budgets to determine Campus-Based 
eligibility for each student 

« Has wide discretion in the amount and composition (package) 
of Campus-Based aid awarded to students with varying levels 
of need 

♦ Could/ and in many cases had to, allow total Campus-Based aid 
to fall short of need unmet by ocher programs 

♦ Had to follow three different sets of regulations governing 
the three Campus--Based Programs. 



The Guaranteed Student Loan (GSL) Program provides the most 
financial assistance to postsecondary students of all the Title IV 
programs. It makes available to students attending eligible 
postsecondary institutions loan funds with which to meet educational 
expenses. The program uses capital provided through private-sector 
banks, savings and loan associations, credit unions, other financial 
entities, and educational institutions. Postsecondary institutions 
"certify" the loan application by providing key information that permits 
the lending institution to determine the applicant's eligibilty and loan 
amount. 

Despite the structural di:!ferences among these five programs, many 
components of their delivery systems are similar, as shown in Exhibit 
1-6. Students apply for financial assistance, providing information 
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EXHIBIT 1-6 COMPONENTS OF THE TTTLE IV STUDENT AID DELIVERY SYSTEMS 
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calculated based on eligibility/ need, enrollment status, and cost of 
education. Funds are disbursed to students, either directly or through 
the institutions. Institutions must reconcile their accounts for each 
program ard report to ED. 

In the Pell Grant and Campus-Based programs, funds are disbursed to 
students through the institutions. (For a small per entage of Pell 
recipients, whose institutions do not participate in the program, funds 
are disbursed directly to students from the Federal government through 
the Alternate Disbursement System.) In the GSL program, students may 
receive their loans either directly from the lenders or from the lenders 
through the institutions. 

The data used to determine financial need ar<» virtually the samo 
across all the programs. Students must provide information regarding 
family income (both taxable and nontaxable), expense', assets, and 
household sire and college attendance. In the GSL program, students with 
a family adjusted gross income (AGI) of less than $30,000 are assumed to 
be eligible for GSL*s. Students with family AGI*s of more than $30,000 
must undergo eligibility determination based on need. 

As noted above, application data are treated differently in the 
programs. In the Pell Grant program, a central application processor 
receives the application data, and adheres to a specific formula for 
determining need. In the Camp -^-Based programs, need may be determined 
by a need analysis service, c by the institution, and the program 
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regulations allow for institutional discretion in adjusting individual 
items to acconunodate specific :student circumstances. In the GSL program, 
institutions mainly certify the students' enrollment status and 
categorical eligibility; the lender handles all other application 
procedures. 

1.3 DATA SOURCES 

Advanced Technology. Inc., engaged Westat, Inc., of Rockville, 
Maryland, to perform field work and provide technical assistance in 
special areas such as sample design. Data for this study were collected 
from three sources: first, from institutions, second, from students and 
parents, and third, from "external" sources which could confirm data 
obtained from the student and parent data collection, as illustrated in 
Exhibit 1-8. In additi^^n, data on Pell Grant recipients were abstracted 
from the Computed Applicant Record (CAR) maintained by the Pell Grant 
central processor. The selection of institutions and students will be 
discussed in detail in Chapter 3. The data collection activities are the 
focus of Chapters 4 and 5. 

The institutional data collection consisted of interviews with the 

directors of financial aid at 297 institutions of postsecondary education 

and a detailed abstraction of information from the financial aid records 
of 2,996 students. 

Institutional Questionnaire (IQ). The Institutional Questionnaire 
was administered to the financial aid administrator during an interview 
at each sampled institution. One of this questionnaire's major purposes 

1-15 ^ * 

^ o 



was to provide information on institutional policies and procedures 
(primarily on need analysis ^ packaging, student eligibility, student 
enrollment status / cost of attendance, and disbursements) that might be 
required to assess institutional compliance with program requirements. 
This information was required to determine the student's correct need and 
to calculate the extent of error. We also used che information to 
identify institutional characteristics that could be correlated with the 
presence or absence of error in the analysis of possible corrective 
actions. We used the infcrmaHion collected during the interview in 
conjunction with information collected on individual students iu the 
Student Record Abstract (SRA). The IQ also included a series of 
questions about institutional quality control procedures to determine the 
current level of institutional quality control and to identify those 
activities which could be correlated to reduced levels of error. 

Student Record Abstract (SRA). The Student Record Abstract was 
used by the field data collectors during the institutional visits. 
Information from student aid files for each of the selected students at 
the sampled institutions was abstracted onto this form. The sections of 
the SRA and the purposes of each were as follows: 

• General Eligibility ~ to determine both the 
institutionally-used and correct student eligibility and 
enrollment status data 

• Poll Grant Program — to record data concerning the Student 
Aid Repor t and val idat ion and to determine bo th the 
institutionally-used and best cost of attendance 

• Campus-Based Programs — to record data concerning need 
analysis* cost of attendance, awards, and specific program 
eligibility 

• GSL Program — to record data concerning need analysis, cost 
of attendance, other aid available/ and loan eligibility 
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♦ Dociimentation — to record applicant-reported valueS/ 
institutional adjustments to the application/ and 
documeiatatioa found in tho students* files 

♦ Disbursements and Repayments/Refunds — to record 
disbursements and related information by program and to 
determine what repayments or refunds were ma'^3 and whether 
they were done properly. 

The student and parent data collection consisted of iu-person 
interviews with the sampled students and/ if they were dependent/ their 
parents • 

Student Questionnaire (SQ). The Student Questionnaire was designed 
to obtain information and documentation to confirm or verify data 
reported on the aid application completed by the student. We used these 
figures in conjunction with other data to determine award errors and 
discrepancies in need calculation. 

Parent Questionnaire (PQ). The Parent Questionnaire was designed 
in parallel with the SQ, but with items referring to the parent(s). If 
the dependency status of independent students was confirmed in the PQ/ 
further questions/ concerning income and assetS/ were omitted. 

External Sources of Documentation (Secondary Data). During the 
conduct of the student and parent interviews/ interviewers obtained 
permission for Westat to receive verifying information on income and net 
worth. Students and parents provided written permission for the release 
of Forms 1040/ 1040A/ or 1040E2 from the Internal Revenue Service and 
forms from financial institutions verifying the value of savings and 
checking accounts at the time of application/ if the total amount of 
those accounts reported to the interviewer was unknown or $4/000 or 
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more. For a sample of 25 percent of the respondents reporting home 
ownership/ local tax assessor's offices were contacted to ascertain the 
approximate market value of the respondent's home or primary residence. 

1,4 LIMITATIONS OF STUDY DESIGN AND METHODOLOGY 

The study design and methodology employed in Stage Two of the Title 
IV Quality Control Project has been thoroughly tested and has proved to 
be both efficient and effective in producing important data concerning 
quality in the various Title IV student aid programs for policy making 
purposes. The methodology produces the most robust results at the 
program-wide l67el. However/ the design necessary to produce these 
results coupled with the data collection and sample size restrictions 
imposed by cost considerations present limitations that must be 
explicitly stated in presenting the study methodology. 

The Title IV Quality Control Project's primary objectives relate to 
measuring error and identifying and assessing the likely effects of 
corrective actions. The first objective/ concerned with measuring error/ 
is a prerequisite to the remaining objectives (which focus on corrective 
actions) and thus drives the study design. A design that maximizes the 
ability to measure and decompose error necessarily focuses on the 
numerous potential error points in the delivery system. Consequently/ the 
greatest portion of data collection resources are dedicated to obtaining 
student application and institutional data related to the deliveiy system 
error points through student record abstraction/ interviews/ and 
obtaining data from banks / Federal agencies and other sources. 
Fundamentally/ the study methodology becomes corroborative: the data 



coll*»ction seeks to obtain data from alternative, more reliable sources 
in order to confirm the validity of ths data originally used to award 
aid. 

The necessary dominance of error measurement in the study design has 
important implications for the ability to identify and assess the likely 
effects of particular corrective actions to reduce error. The first 
implication limits the identification of corrective actions fo: analysis 
to practices tiiat exist at institutions. Second, given realistic 
resource constraints , data collection focused on institutional practices 
and characteristics and was limited to a small set of data that are 
hypothesized to relate to variation in error. Actively searching for 
other practices and characteristics would require different methodologies 
such as case study and process assessment techniques. In addition, no 
cost data are available for corrective actions, since this would also 
require different data collection techniques. Third, the sampling 
requirements for a national error study necessitate sampling a large 
number of institutions with a relatively small nxxmber of students at 
each. For these reasons, tliis sampling design maximizes the precision of 
program-wide error estimates. However, the number of students that can 
be sampled and the depth of the collection of institutional data are 
minimized due to cost and burden considerations. 

Two other factors pose limitations for the study results. The data 
themselves pose important limitations for both characteristics associated 
with error and identification of corrective actions. The relative 
frequency and variation of certain types of error - particularly 
institutional error - virtually preclude meaningful analysis. For 
example, tho occurrence of individual errors may be relatively 



infrequent, despite the magnitude of the payment consequences/ and 
therefore not yield an adequate number of observations for analysis • In 
addition, error may not vary adequately across instrumental variables 
(e.g., institutional practices) to produce conclusions concerning *.he 
relationship between the dependent and independent variables. 
Frequently, error may be varying by other variables, for which we are 
unable to control due to the data collection focus. 

Lastly, the sample size poses a limitation for analysis. A sample of 
approximately 300 institutions and 3,000 students will yield error 
estimates at the program-wide level that are sound for policy making 
purposes. However, precision of error estimatesi at this sample size (a 
function of cost) drops as error is decomposed into sources and 
especially individual errors. The combination of the relative 
infrequency of certain individual errors and the implication of the 
overall sample size at this level effectively limit the analyses that can 
be couducted. 

Despite these limitations, the design methodology effectiv.^ly 
produces robust, important policy making data concerning quality in the 
Title IV scudent aid programs and powerful data concerninq corrective 
action initiatives at high ±evels of aggregation. 

l.b RELATIONSHIP TO OTHEfe VOLUMES OF THE FINAL REPORT 

The final report on Stage Two of the Title IV Quality Control Project 
consists of this report plus two other numbered volumes and an executive 



sxuiunary. The executive summary includes material from all three numbered 
voliunes. 



Volxxme I of the final report/ Findings , presents the principal 
findings regarding the level and sources of error and the most 
significant individual errors and groups of error in the Pell and 
Campus-Based programs and the GSL certification process. The analysis in 
that volume includes institutional and student characteristics that are 
associated with error. 

Since definition and measur^^ment of error is closely associated with 
the substantive findings, we have discussec definitions and mearurements 
in detail in Volxxme I; this discussion is sximmarized in Chapter 2 of this 
voliune. 

Volxxme II^ Corrective Actions , is based on Volume I. In it we 
recommend those corrective actions which appear to be most promising in 
reducing the amount of error in the various programs. 

In this report, Volxxme III, Procedures and Method s, we present the 
methodology used in conducting the study, including sampling^ data 
collection a data processing, and the level of nonresponse and 
estimates of variance and their effects on the sample. 
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ERROR DEFIWinON AND ERROR MEASUREMENT 

In this chapter we will discuss how we have defined error in Stage 
Two in the context of Stage Two objectives and how these error 
definitions relate to previous studies. We will review the major 
objectives and findings of the previous studies as they have influenced 
our current emphasis on errors that affect the redistribution of program 
funds. We will present and discuss the measures of error by program, as 
they were developed to reflect specific program structures. Finally^ we 
will discuss aggregation of error into program-wide estimates of error. 

As stated in Chapter 1/ the objectives of Stage Two of the Title IV 
Quality Control Project are to measure error in five Title IV programs; 
to determine the extent to which error has persisted in the programs and 
the existence of patterns of error; to assess the effectiveness of prior 
corrective actions in terms of both error removed and residual error; and 
to recommend corrective actions to further reduce error and maintain 
program intent. These objectives have guided the study design for Stage 
Two/ and shaped our approach to error definition. They are a direct 
result of the objectives and findings of earlier QC studies. 

The incidence of errors iu Stage One was similar to the findings from 
the Pell QC Study. Student misreporting was a major source of error. 



Because all programs have similar application processes and procedures at 
the institutional level, a Title IV-wide focus to both error measurement 
and management corrective actions is appropriate. 

2.1 ERROR DEFINITION 

Our approach to error definition and error measurement in Stage Two 
defines error as the difference between need or award calculated using 
data reported by the student and/or used by the institution - referred to 
as baseline data - and "best value" data, which is data obtained during 
the course of data collection that are considered the most accurate and 
reliable data available. Our methodology is designed to do this by 
collecting data from students, parents , institutions, and external 
sources to confirm the values used in calculating need and award. In the 
absence of such confirmatory data, for any given item, the value reported 
by the student is accepted and used for analysis purposes as the best 
value • 

Stage Two is unique in that it measures errors in all five (Pell, 
three Campus-Based programs and GSL) Title IV programs. However, 
differences among the programs require the use of different error 
measures and avoidance of comparisons of data that are not comparable. 
For example, it is inappropriate to compare program-wide payment error in 
the Pell Grant program with awards in excess of need in the Campus-Based 
programs, since the former is directly related to the distribution of 
Federal program funds, while the latter is not. Rather, we have stressed 
cross-year compari.'ions as relevant since U / provide indications of the 
trends in program error estlm.' .es. 
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2.2 MBASOTEMENT OF ERROR 



Measurement of error in the Pell, Campus -Based, and GSL programs 
requires the use of several conceptually distinct error definitions due 
to the unique characteristics of each program. For example, because Pell 
is a formula-driven entitlement program, changes in a given reported da'-a 
element will have a knovm effect on the applicant's Student Aid Index and 
Pell Grant award at a given enrollment status and cost of attendance. 
Thus, such changes, and therefore error, can be modeled precisely. 

The characteristics of the Campus-Based programs require a distinct 
and different approach to conceptualizing and measuring error. The 
purpose of Stage One was not to find common ground with Pell, but to 
address unique aspects of the Campus-Based and GSL programs, where errors 
do not always translate directly into award changes or actual cost to the 
Federal government. For example, changes in Campus-Based need may occur, 
but because institutions frequently do not meet full need, need changes 
do not necessarily result in award changes. However, even if awards were 
to change, the Campus-Based funds, allocated to institutions for 
distribution to students, would be reallocated to other students or to 
more students, ra^'her than returned to the Treasury. 

In the Campus-Based programs, need analysis (e.g.. Family 
Contribution Schedule and need analysis service formulas ) performs a 
function much like the Pell formula, but the resulting expected family 
contribution (EFC) does not determine an award; rather, the EFC is used 
by institutions as an input to discretionary packaging algorithms (formal 
or informal) to meet a fixed or variable portion of need with a grant, 
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loan and/or work. Thus, the effect that changed in student- reported 
data, for example, will have on awards can only be estimated. Analysis 
of error in Campus-Based need is a necessary intermediate step and an 
important measure of the impact of student errors and other errors in the 
programs. However, because of the intervention of institutional 
discretion, need error cannot be used as a proxy for error in awards / 
since need changes often do not result in dollar-for-dollar changes in 
awards . 

The regulatory definition of error, defined as awards made in excess 
of need, is not an entirely satisfactory measure; it underestimates the 
likely impact of need changes because few institutions meet full need. 
For example, a student whose Campus-Based need falls from $10,000 to 
$5,000 and received a total of $5,000 in Campus-Based aid would have no 
award in excess of need and, thus, no error, despite errors having caused 
a substantial need change. 

Thus, an additional measure of error must be used in assessing 
quality in the Campus-Based programs: distributional error. This 
measure uses packaging algorithms that are developed from actual 
institutional policies and constraints to repackage Campus-Based awards 
for students with need changes. It has the effect of simulating what 
institutions vould have awarded, all other things being equal, had best 
values been available when aid was originally awarded. Distributional 
error more closely estimates the total impact of errors in the 
Campus-Based programs. 
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Measurement of error in the GSL program presents other methodological 
problems • Unlike the other Title Iv programs, inscitutions play a 
limited role in the GSL program: certifying the amount for which a 
studen^ is eligible. Program limits, students who apply for specific 
amounts, and lenders and guarantee agencies jointly determine the actual 
loan amount. Often, institutions may not know the exact loan amount, or 
if the student even completed the loan process and received a loan. 
Therefore, because the study design focused on the institution, 
measurement of error focuses on certification rather than other, 
non-institutional aspects of the GSL delivery system. 

2«2«1 Error Measures 

The above-mentioned considerations require five different aggregate 
error measures for the three programs. Programmatic and measurement 
differences make these error measures incomparable, since identical 
errors translate into payment consequences at different rates in each of 
the programs. Thus, the relevant comparison for each error measure is 
with studies in prior years. These error measures are as follows; 

^ Pell program-wide payment error is a measure of differences 
between actual awards generated from reported data and best 
awards using best data. This is a measure of deviation from 
quality in the program. 

• Casipus-Based need error is a measure of the impact of 
student reporting error and certain institutional errors in 
Ccmpus-Based need. This is computed by comparing reported 
ne€)d used by institutions to package awards with best need 
calculated using best values. Need is the simple difference 
between the cost of attendance, and family contribution and 
other resources. 

♦ CaMpus-Based awards in excess of need is a measure that 
approximates the regulatory concept of error in that o ^ly 
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those need changes that fall below award are considered in 
error. 

• Campus-Based distributional error is a measure that more 
closely approximates the likely payment consequences since 
all need changes are considered and repackaged to simulate 
institutional packaging. 

• Guaranteed Student Loan certification error is a measure of 
the aggregate change in certifications when best cost of 
attendance^ need and other aid are substituted for reported 
data. Certifications are capped at the 1985-1986 program 
limits of $2/500 for undergraduates and $5^000 for graduate 
students. 



These error measures will be decomposed into two types of error: 
student reporting error - which is used to motivate corrective actions 
rather than assign responsibility - and institutional error. The 
definitions are as follows: 



• Student reporting error is the result of recipients 
providing inaccurate data at the time of application and 
subsequent to .it. This decomposition is silent on whether 
the error was conscious or inadvertent or ichether it was true 
at the time and sixbseguently changed. 

e Institutional error is the result of institutions usinc 
incorrect data in awarding^ processing or disbursing aid and 
includes errors that affect student need/ ''categorical" 
eligibility/ disbursements/ and required procedures (mostly 
collection of documentation). 



These errors will be decomposed further to identify significant 
individual or groups of errors as a basis for corrective actions 
analysis. 



2.2.2 Effects of Best Values Selection on Srror Measurement 



The instrtiments and data sources that constitute data collection 




provide the foundation for selecting the best values for each application 
item required for determining eligibility for each program in the study. 
Since error is based on the difference between values reported on the aid 
application (or used by the institution) and best verified values, error 
measurement is only as accurate as best value selection. By the time 
data collection ended, many questions had been asked to confirm or deny 
the veracity of the values shown on the application for financial aid. 
These application items fall into seven major categories: 

• Dependency status 

• Tasabla income and taxts paid 

• Other nontaxable income 

• Offsets to income (including household si^e, number in 
college, employment expenses, and other expenses that reduce 
available incom-) 

• Assets and debts 

^ Other educational benefits 

^ Dependent student's income and assets. 

These student and parent data are used to derive the Pell Student Aid 

Index (SAD and the expected family contribution (EFC). In the 

Campus-Based programs, and under certain circumstances GSL, the SAI and 
the EFC are two of the indices on which awards are based. 

Since costs of higher education vary greatly due to differences in 

tuition, fees, and living expenses, we also confirmed the accuracy of the 

figures used by the institution to determine cost of attendance. In 

addition, we checked various student eligibility requirements, program 

requirements, student enrollment status records, and disbursements^ as 

required by the regulations for the Pell/ Campus-Based, or GSL programs, 
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The multiple data sources used in the st.udy meant that many different 
values could have emerged during tha co»;^rse of checking on application 
values of students and parents. If these values were consistent/ best 
valuo selection was a simple matter. If these values differed^ however/ 
a method was required to determine the best values. The best value was 
the one that was documented and c<yne from the most reliable source. This 
was determined by merging the data from the various sources and selecting 
the best value using a SAS program designed for that purpose. The 
program was designed to select a value from a hierarchy of sources. 
Because the program selected the most reliable documented source/ it 
selected the **best" or most reliable value available. In all cases / 
however/ the program defaulted to the value reported by the applicant if 
more reliable data were not available. Refer to Chapter 6 for more 
information on best value selection. 

2,3 CC»<PDTATION OF ERROR 

The purpose of this section is to provide the reader with sufficient 
general knowledge to understand how we arrived at the error figures. 
These general computationa.l procedures were developed specifically for 
this study and are based on the data available. They have been expanded 
into the necessary technical specifications in the software developed for 
error computation. 

2.3.1 Pell Error 

The Pell award is a function of SAI, cost of attendance/ and 
enrollment status. In addition/ students must meet certain eligibility 
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requirements to receive an award. Three separate Pell awards are used in 



calculating Pell errors: 

♦ Best Pell Award: The award that should have been disbursed 
if there were no errors conunitted at any point in the award 
process. It is calculated using all best data, including 
best SAI, best cost of attendance, best enrollment status, 
and best eligibility. 

♦ Best Institution Pell Award: The award that should have been 
disbursed if there were no institution mistakes committed 
during the award process. It is calculated using the SAI 
reported by the student, and best institution data/ including 
best cost of attendance, best enrollment status, and best 
eligibility* 

♦ Actual Pell Award: The award actually disbursed by the 
institution. It is equal to the total of Victual and planned 
disbursements . 

Comparisons of these three awards yield total, student, and institution 
errors in the Pell program: 

♦ Overall Error: The amount by which the award disbursed 
differed from the award that should have been disbursed. It 
measures '^^viation from quality in the program and is equal 
to the Ac^ al Pell Award minus the Best Pell Award. 

♦ Institution Error: The amount by which the award disbursed 
differed from the award that should have been disbursed if no 
institutional mistakes were committed. It equals the Actual 
Pell Award minus the Best Institution Pell Award. 

♦ Student Error: The impact of recipients providing inaccurate 
data at the time of application or subsequent to it. It 
equals the Best Institution Pell Award minus the Best Pell 
Award. 



Both overall and institution errors are relatively straightforward 
concepts. Student error, however, is somewhat more complex. By 
calculating student error as the difference between the Best Institution 
Pell Award and the Best Pell Award, we are measuring the impact of 
student errors in the SAI, holding institutional parameters constant at 
best values. Measuring student error in this way ensures that in an 



individual case/ student and institution errors do not exceed the amount 
actually disbursed. For ineligible students who snould not have received 
any award/ student error will be zero while institution error will equal 
the amount actually disbursed. Special condition filers are also defined 
to have no student error by setting best SAI equal to reported Ski. 
Student error cannot be measured for special condition filers because 
their awards are based on different data elements than the other 
recipients • 

2.4 CAMPUS-BASED NEED ERROR 

The general formula for computing need is; 

Need = Cost/Enrollment status - Pell - EFC - Other Known Aid - (GSL 
Resource) 

Cost/Enrollment status refers o the appropriate cost of attendance 
at a given enrollment status level. From that we subtracted the amount 
of the Pell Grant/ the expected family conti oution, the amount of other 
known non-Federal aid (e.g./ state/ institutional/ or private loans or 
grants)/ and the GSL resource. For Campus-Based need/ a GSL resource 
must be subtracted as another source of aid if the GSL was certified by 
the date that the Campus-Based aid was packaged. The GSL resource is the 
amount borrowed. If AGI is $30/000 or less, the GSL resource is the 
amount/ if any/ by which borrowing ci*wc*»ds EFC (i.e./ if AGI is $30/000 
or less the EFC may be borrowed and so should not be considered a 
resource in computing need). If the GSL was certified aiTte^" C<impus-5aced 
> aid was awarded/ we consideired the GSL resource to be zero. We made this 
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assumption to avoid holding institutions responsible for a GSL resource 
that they may not have known about or reasonably ancicipated at che time 
that the Campus-Based aid was packaged. 

To compute error, we calculated need in two ways. One way used 
reported values and the second used best values. Both calculations used 
the general formula above. The difference between these two calculated 
need figures is overall need error in its simplest form: 

Overall Need Error = Need Based on Reported Data - Need Based on Best 

Data 

For both need based on reported values and need based on best values, 
we followed specific procedures concerning the source of the values used 
in the calculation. For reported need, we used the values actually used 
by the institution for cost/enrollment status. Pell Grant, EEC, other 
known aid, and GSL resource. 

Calculation of need based on best values is considerably more 
complicated than reported need. One reason is that we must calculate 
best need differently for overall error, student error, and institutional 
error. While we used best values for both student and institutional 
variables in best overall need, for best student need we had to separate 
out the effects of student/parent misreporting. For best institutional 
need we used best institutional values, but kept student values constant 
at the reported level to separate out the effects of institutional errors. 
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to be taken into account at the time of Campus -Based packaging or GSL 
certification and any changes in Peii couid significantly affect the 
remaining need. Best Pell was used in the Campus-Based need calculations 
whether or not the student received a Pell award. Pell was required to 
be counted as a resource by schools whether accepted or not by the 
student. 

For other known aid, however , the amount listed in the student's file 
was used in best award computation. This is because other known aid 
consists of non-Title IV aid for which we had no means to calculate what 
changes/ if any, would be made as a result of changes in application 
values. An other aid error can occur, however, if an institution fails 
to include the total amount of other aid found in the student's file in 
calculating need. 

If Campus-Based aid is packaged after GSL (this determination is 
made based on questions in the SRA and the timing of award amounts), then 
the best value of the GSL resource was considered in determining 
Campv -Based need. We assumed that a GSL is first used to offset the 
family conx.ribution to the extent possible and that only the ainount 
remaining is considered as a resource for Campus-Based award. The "best** 
GSL resource is the minimiim of the GSL award actually received by the 
student and the determination of "best" GSL need for that student; cor 
students with AGI*s under $30,000, the best EFC is subtracted from this 
amount to arrive at the best GSL resource. Defining the best GSL 
resource in this manner avoids double counting the error associated with 
a given case and prioritizes the aid in the same manner used by the 
institution. 
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When calculating institutional error/ we used the best value for 
cost/enrollment status and the values listed in the file for other known 
aid. For Pell, EFC, and GSL resource/ we used best values for 
institutional components reported values for student components. 

Together/ this yields a figure that indicates what the institution would 
have done if it made no errors when using the student reported values. 
This is then subtracted from a need figure which was calculated using all 
reported values. 

When calculating student need error/ best institution need is 
subtracted from best need and the difference is added to reported need. 
(Need detennin*?^d using all reported values.) This yields a student need 
value that reflects what would have been used by the institution if the 
student made no errors, assuming that the institution* s behavior remained 
unchanged. This io then subtracted from a need figure which was 
calculated using all reported values. 

One of the hallmarks of the Campus-Based programs is the discretion 
granted to financial aid administrators zo adjust applicant information 
to reflect changed circumstances that are not considered in the EFC 
generated by the need analysis. We anticipated these adjustments to 
individual items in our design of the SRA and in our selection of best 
values. If an item adjustment is explained or documented in the 
student's tile, it is used as the best value and there is no student 
error. If the adjustment is not explained/ we considered the omission an 
institutional error that affected determination of need/ with error as 
Che difference between the unexplained adjusted value and tho, reported 
value. When calculating institutional error/ we used the unexplained 
folu* vith the institution values/ but the student reported value with 



the other institutional best values. In addition, student error would be 
calculated using the difference between the best value and the student 
reported valufe. Overall error for an unexplained adjustment is the best 
value minus the unexplained adjusted value, because that is the value the 
institution actually used. 

In addition to or instead of adjusting individual items, institutions 
may adjust the EFC either after calculation by the need analysis service 
or by specifying a variety of institution options in the calculations of 
EFC performe'^. by the service. A review of the SRA data on EFC has 
revealed thes^j practices to be widespread. Furthermore, the adjustments 
to EFC made by the institution are often impossible to replicate, either 
because they are outside the need analysis formula or because the details 
on options specified to the need analysis service are not available from 
the SRA. 

In order to mor^ properly assess the change in EFC which results from 
error in the detailed application items or EFC components, we calculated 
best EFC for all cases as the sum of the reported EFC and the difference 
between two calculated EFC values. The first EFC value was calculated 
using all be^st values (treating explained adjustments as best values). 
The second EFC value was calculated using all reported values. Thus, 

Best BFC s ((EfC calculated using best values) 

- (EFC calculated using reported values)) 
•f Reported EFC 

The following example should clarify this procedure. An AGI error 
exists such that EFC calculated using ai; best values equals $5,000, 



while EFC calculated using reported values equals $4,500. Thus, the AGI 
error leads to a need error of $500. While the EFC calculated using 
reported values equals $4,500, the E?C value used and reportec' by the 
institution equals $3,000. For study purposes, the best EFC would then 
be equal to $3,500 = (($5,000) - ($4,500)) + $3,000. This indirect 
method of determining a best EFC has the effect of controlling for 
legitimate adjustments made to the EFC by institutions and not captured 
by our data collection. Thus, such adjustments are not counted as errors. 

If the need analysis system used by the institution was CSS, ACT. 
Pell FC, or SAI (as it was in nearly 94 percent of the cases) we used 
that need analysis system. For the remaining cases, we used the next 
closest need analysis system, which, in most cases, yielded an EFC within 
$10 of the EFC generated usincj the less frequently used system. (These 
$10 discrepancies were usually due to different rounding conventions.) 
Thus, we established a $10 measurement tolerance for EFC such that 
differences of $10 or less were considered zero. 

2.5 GSL CERTIFICATICH ERROR 

As previously discussed, we have not reported GSL need errors in this 
volume. The reason for this is simple: GSL certification is GSL need 
capped at the annual program limits of $2,500 for an undergraduate 
student and $5,000 for a graduate student. Uncapped GSL need error has 
no value in helping us understand the causes of error or suggesting 
corrective actions, since certification is always capped. Nonetheless, 
we did compute GSL need as an interim step in determining GSL 
certification error. 



The comput::itional procedures for GSL need are virtually the same as 
those used .for Campus-Based need. Thufw rather than reiterating our 
discussion of need error for the GSL program, we will highlight the 
differences ' 

The basic formula for GSL need error is the same as that for 
Campus-Based need error / reported need minus best need. GSL need is 
determined the same way as Campus-Based need except we stibtracted 
Campus-Based award rather than tne GSL resource. Thus: 

Need = Cost/Enrollment status - Pell - Other known aid - Campus-Based 
Award 

Campus-Based award is subtracted only if Campus-Based aid was 
packaged on or before the date of GSL certification. The Pell Grant 
subtracted may be an actual or estimated amount. Reported need figures 
for GSL are taken from the application and certification form. These 
reported values are those actually used by the institution. 

Like Campus-Based need, determining the components of ^SL best need 
is more complicated than reported need. Best overall need is 
straightforward, and uses all best student and institutional values. For 
institutional need, best values are used for institutional components and 
reported ones for student components. When calculating student need, 
best institution need is subtracted from best need and the difference is 
added to reported need. In both cases, these calculated best values are 
stibtracted from the need values calculated using all reported data. 



The amount of Campus-Based award factored into best GSL need depended 
on the packaging dates. We included in this amount any Campus-Based aid 
that the institution had packaged by the certification date, holding 
institutions responsible only for amounts that were known at that time 
and should have been included. Noa-Federal other known aid was again 
based on the values listed in the student's file since calculating best 
values for other aid was not feasible for the reasons discussed above. 



The discretion granted to financial aid administrators to adjust 
items used in EFC applies only when a Campus-Based need analysis system 
is used. Practically speaking, it is only of concern for GSL if the AGI 
is greater than $30,000, since EFC is assumed to be zero for GSL when AGI 
is $30,000 or less. 

In general, the procedures used for fe'alculating GSL EFC when AGI 
exceeds $30,000 are the same as those for Campus-Based. Best EFC is the 
difference between EFC computed using best values and EFC computed using 
reported values, added to the actual EFC reported on the GSL 
certification. However, if the GSL Tables were used, EFC could be 
calculated directly because no adjustments are allowed to be made to the 
GSL Tables. Also, if the student received Campus-Based aid, the 
Campus-Based EFC's were used in GSL. 

Once the reported and best GSL need figures were calculated, they 
were converted into reported and best GSL certification. The reported 
need frequently was capped at appropriate program limits by the 
institution in the maximxxm loan eligibility category on the certification 
form. If it was not, we checked reported data on the year in college and 
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presence of a B.A. degree. The reported GSL certification for an 
undergraduate was $2/500 (or $5,000 for a graduate student) or the need, 
whichever was less. Best need was capped at program limits in the same 
way. This yielded values for reported GSL certification and best GSL 
certification. Error was measured as follows: 

GSL Certification Error = GSL Reported Certification - GSL Best 

Certification 

In order to estimate error in the GSF; program we focused on the point 
in the delivery system that institutions certify students* eligibility 
for loans and determine the maximxam loan amount. However, because not 
all students borrow the maximxim amount, and because students pay these 
loans back over a payment period of up to 10 years, GSL 
overcertif ications is not an accurate estimate of cost to the 
government. ED is responsible only for interest payments while these 
students are in school or other deferment periods, the special allowance 
subsidiary to li^nders, and for the remaining balance on defaulted loans. 
Since some of these costs are tied to the interest paid on U.S. Treasury 
notes (T-'Bills), costs vary substantially as the rate of interest rises 
and falls. On avera^g^, costs per dollar loaned ranged from $.342 with 
T-Bills at 5.5 percent zo $.676 with T-Bills at 10 percent. These 
figures were provided by the Department of Education and represent their 
estimate of the low and high range of net cost per dollar loaned. ED's 
best estimate of costs is $.437 per dollar loaned based on a T-Bill rate 
of 6.6 percent. In order to estimate the costs to the government of GSL 
overcertif ication, we used the average rate of borrowing per dollar of 
certification (84 percent) to translate overcertif ications to loan 
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amounts and multipled those amounts by the cost ranges* 

2*6 CAMPUS -BASED PAYMENT ERROR 

Campus-Based need errors causa overpayment error only when need falls 
below the amount of aid awarded* 



The basic formula for payment error is: 

Payment Error s Campus-Based Aid - Best Campus-Based Need 

Since overpayment errors can occur only when best need is less than the 
amount of aid received/ payment errors are always positive. In 
calculating student and institution payment errors, best Campus-Based 
student need or best Campus-Based institution need were substituted for 
best Campus-Based need. No matter what the magnitude of need errors, 
need was always capped at a minimum of zero so pa^niient error could not 
exceed the amount awarded* 

In the formulas for payment error, the value used for Campus-Based 
aid was usually the amount of aid accepted by the student from all 
Campus-Based aid sources* Some exceptions to this policy have been 
made* For SEOG and NDSL aid, for example, we used the amount of all 
disbursements if it was less than aid accepted, to reflect the fact that 
in some cases, awarded aid was later reduced by the institution* For 
CW-S aid we used the amount accepted or the amount disbursed, whichever 
was greater. When computing disbursement error payment consequences, we 
always used the amount of aid disbursed. 




2-19 



55 



2.7 CAMPUS-BASED OISTSIBUTIOMAL ERROR 



For distributional error we estimated the payment consequences of all 
need changes^ not only those that fall below award. These payment 
consequences measure the anticipated redistribution of funds if packaging 
guidelines used by the institution were followed for best need. We used 
two types of measures in our formula for the calculation of payment 
error. Firsts we looked at the initial proportion of total aid 
represented by each of the Campus-Based programs for each student. We 
retained these proportions in our repackaging of aid. Second, we 
included several questions in the Institutional Questionnaire (IQ) to 
allow us to replicate institutional packaging practices and constraints. 
We selected eight of the most frequently applied general practices and 
constraints for use in repackaging. Five of these applied to the three 
Campus-^Based programs and three applied to all aid, by undergraduate or 
graduate status. The three packaging constraint questions asked 
concerning all aid^ for undergraduates and graduates, respectively, were; 

♦ Limit total award to $ 



• Limit awards to students with at least S of need 

• Always have $ of unmet need. 

The five constraints that institutions were asked if they place on each 

of the three Campus-Based awards were: 

• Maximtim EFC 

® Maximum dependent parent's AGI 

• Maximum independent student's AGI 

• Minimum avard 
Q • Maximum award 
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For each question answered "yes/* the institution was asked the dollar 
ciTtCunt or percent applicable* The repackaging measure only applied these 
constraints to students with need error. By definition/ students without 
need error had no distributional error. Also, if the school violated its 
own packaging guidelines, distributional error was capped at need error 
to ensure that error was not attributed to schools for not following 
their packaging policies. 

2.8 LIABILITY 

Previous quality control studies have evaluated the broad delivery 
process/ including in error definitions discrepancies that are not 
strictly regulatory violations. The goals of the studies have been to 
evaluate deviation from the intent of various aspects of the delivery 
system. For example, the studies have explored whether estimated or 
prospective applicant data are accurate predictors in an attempt to 
evaluate the effects of such data on the distribution of program funds to 
students . 

In Stage Two, we continue to distinguish between liability according 
to program regulations and tho measurment and analysis of error that will 
accurately reflect deviation from quality in the delivery of Title IV 
funds. Our study design focuses on errors leading to measurable payment 
consequences/ and particularly those errors which prior studies have 
indicated occur with a degree of frequency. 



2-21 57 



This is consistent with the study objectives, stated in Chapter I, 
which encompass the determination of error in the Title IV programs, 
patterns of error across the programs^ and the effects of prior and 
potential corrective actions. These objectives require an approach to 
error measurement that focuses on the broad functions of the delivery 
systems and the achievement of program intent. 

Liabilityr as a narrower approach to error measurements is an 
important subsidiary concept in error measurement, but does not 
constitute a comprehensive reflection of deviation from quality in the 
Title IV programs. 

2.9 AGGREGATICttI OF ERROS 

The Stage Two study design maximizes the precision of estimating 
aggregate error for each of the three Title IV programs, Pell. 
Campus-Based, and GSL. The design provides a national sample of 
recipients and thus a rich recipient data base. Institutional data are 
collected to provide information with which to recalculate recipient 
awards and institutional characteristics for analysis. The sampling 
design selects a small number of students from each institution among a 
large number of institutions. This design provides adequate precision 
for aggregate error estimates. 

2.9.1 Prograia-wide Srror Aggregaticn 



We developed program-wide estimates — totals, averages, and 
proportions — from individual studem: records using information from the 
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sampling procedures and certain program information* Each student record 
had three estimation weights, one for Pell program estimates, one tor GSL 
estimates/ and one for Campus-Based estimates* These weights differed 
across students in each school and across institutions for each program. 



The general form of the estimators is as follows: 



Totals: Xi » ET Vtjn Xtji 

Means: » tCT Wi j ^ Xu c 3 / tEE 4 ^ 3 

Proportions: . nx Vtifeitjfel / 4 i 3 



The i subscript refers to students, 2 refers to schools, and k refers 
to the program. In the formulas above, w. refers to the weight for 
student i at school 2 program k; X.. is the value for a 

characteristic or variable (in this study the value of a particular 
error) for student i at school 2 program k; and is a variable 

1 J K 

which equals 1 if the student possesses a particular categorical 
characteristic (the occurrence of a particular type of error) and 0 
otherwise. (For students in only one program, the weight variable takes 
on a value of zero for the other programs.) 



We made estimates for subgroups of the overall populations by 
selecting only the records of the desired subpopulations or subgroups; 
these records included the weights and student characteristics. The 
weights were developed as the product of two components: the inverse of 
the selection probability for all programs, and the inverse of the 
response rate for students with similar characteristics. 



Estimators using only the inverse of ^the sampling probabilities are 



for this study were based on the probabilities associated with the 
following sample selection steps: 

• Selection of the geographic cluster 

• Selection cf the school within the cluster 

o Selection of the branch for schools with multiple campuses 

• Selection of the student from the appropriate program (Pell, 
GSL, or Campus-Based) list at the institution. 

Probabilities at the final stage reflect the possibility that certain 
students had more than one chance of selection if they participated in 
more than one program. 

The development of- the second component, the adjustment for 
differential nonresponse rates, is discussed in Chapter 3, Section 3,5. 
This factor adjusted for any bias that would have arisen if the expected 
values of the characteristic of interest differed for the subgroups which 
had different response rates. It did not adjust for bias introduced by 
differences in. the expected values of the characteristic of interest 
between respondents and nonrespondents, 

2.9.2 Analysis of Overlapping Errors in the Title IV Programs 

Many of the potential errors in the Title IV programs involve the 
sajne funds, and many errors individually cause a whole award to be 
considered in error. Therefore, we developed methods to prevent multiple 



unbiased, minimum-variemce, efficient estimators* 
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counting of error in the individual cases on which program-wide estimates 



were based. 
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First, we analyzed individual student and institution data items to 
determine their specific contributions to potential errors in the 
calculation of need, program eligibility, award calculation, and 
disbursement. We measured the change in need attributable to 
misreporting of income, assets, expenses, family composition, or other 
factors used to calculate need. No individual application error, which 
affects the calculation of the student ' s need, can automatically 
invalidate an entire award. However, the interactions of changes in 
various elements of the need formulas (from reported to best values) are 
complex, and total need change can be more or less than the sum of need 
changes attributable to individual errors. 



Therefore, the effects of changes in individual need-formula 
components were calculated by substituting in the formula only the best 
value for that item, retaining reported values for all the rest. Total 
need change was calculated separately by replacing all reported values 
with best values. For example, in a given case, a change in AGI may 
appear to affect a student's need, but may be cancelled out (or, 
conversely, magnified) by a change in family size, depending on the 
direction of each change. 



Institutional errors in need calculation also affect the amount of 
need, but do not invalidate an entire award. Institutional errors, also, 
were calculated singly by substituting a single best value/ while 
retaining reported values for all other elements of the need formula. 
Then we replaced all institutionally-determined elements of the formula 
with best values while retaining reported values for all application 
items, and obtained total institutional er^'or figures. 
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SAMPLING 



The Stage Two sample was designed to represent all students 
participating in one or more of the Title IV programs during the 1985-86 
school year. The development of an efficient sample design involved 
several stages, resulting in a self -weighting probability sample. In 
addition. Appendix A contains a discussion of estimates of statistical 
precision and components of variance that were considered in the sample 
design. Chapter 7 of this volume includes a discussion of sampling error 
in the actual sample. 

Based on the results from Stage One and the precision requirements of 
Stage Two and available funds, we chose a target sample of 300 
participating institutions and approximately 3,200 students. 

3.1 FEATURES OF THE SAMPLE DESIGN 

The sample design specified a series of procedures in order to ensure 
a nationally representative sample of Title IV recipients, as shown in 
Exhibit 3-1. Because there was no sampling frame or list of all Title IV 
recipients from which a simple random sample could be drawn, we first 
constructed a master sampling frame of institutions participating in the 
Pell Grant and/or Campus-Based aid programs, and a sampling frame of 
w-JL-only institutions. We had to construct such a frame from existing 
separate lists of institutions participating in the Title IV programs. 



CONSTRUCT SAMPUNG FRAME 
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EXHIBIT 3-1 

MAIN FEATURES OF THE STAGE TWO SAMPLE DESIGM 
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We then divided the contiguous 48 states into 100 geographic sampling 
clusters based on zip codes from the master Fell-Campus-Based sanipling 
fo'ame. Cost limitations required that both institutions visited and 
students interviewed be clustered geographically to minimize the time and 
expense of travel for both institutional data collectors and interviewers 
of parents and students. 

The next stage of sampling was the selection of institutions within 
sample clusters. Because of operational and cost constraints it was 
desirable to select a single institution sample, rather than separate 
institution samples for each of the programs within Title IV. This 
required the development of a measure of size for sample selection that 
would result in the most efficient institution sample under the 
one-sample constraint. Two hundred and eighty-nine institutions were 
selected from the 100-cluster master frame, with probability proportional 
to a measure of size. Fifty institutions were also selected, with equal 
probabilityr from the GSL-only frame. Although our overall target was 
300 schools, we deliberately oversampled tfcie GSL-only schools because of 
the poor quality of data on the GSL tape, i.e., numerous institutions 
were out of business or could not be located. 

Finally, students within institutions were sampled from lists of Pell 
and Campus-Based recipients and GSL certifications, obtained by field 
staff during the institutional visits. Since we had distinct sample size 
goals for each program, separate sampling lists were required for each 
program. Where a student participated in more than one program, the 
sample from which the student was actually drawn was used for determining 
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the sampling response rates? all programs from which the student received 
aid were considered in the analysis of the data. 

3.2 SAMPLING FRAMES 

The Stage Two sample design specified a clustered multi-stage 
sampling procedure involving, first, the selection of postsecondary 
institutions and, then, of students within the selected institutions. To 
carry out the selection of institutions / it was necessary to obtain or 
construct a complete list of institutions in the universe of interest; 
that is, those participating in one or more of the Title IV aid 
programs. After -sample institutions were identified it was necessary to 
obtain lists of students in the programs of interest. This section 
describes the approach followed in the construction of these sampling 
frames . 



3 •2.1 Institutional Sampling Frame 



The ideal institution sampling frame for the study would consist of 
all postsecondary schools which either had participated in Pell or 
C/jnpus-Based programs in academic year 1985-86 or had certified one or 
more students for a GSL for the same time period. Also, the ideal frame 
would have a measure of size - the number of participants or 
certifications or at least enrollment - for each program present at each 

9 

institutxon. 
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The sampling frame was constructed by merging four universe files we 
obtained through the Department of Education 

• Institutions receiving Campus -Based funds in academic year 
1984-85 

• Institutions with students receiving Pell grants in academic 
year 1984-85, and 1985-1986 

• Institutions with GSL certifications in academic year 1984-85 
or earlier. 

The first step in constructing one sampling frame from the universe 
files was to determine what institutions from each file were eligible for 
inclusion in the sampling frame. The eligibility requirements for 
institutions for each file were: 

• Pell: 

— Eligibility code equal to "1," indicating that the 
institution was eligible 

EiKClusion of institutions whose students may receive Pell 

Grant funds only through the Alternate Disbursement System 

• Campus -Based: 

~ Eligibility code equal to blank, indicating that the 
institution was eligible 

The institution applied for and received funds for at 

least one of the three Campus -Based programs for academic 
year 1984-35 

— Codes for institution type a.id control were valid 

• GSL: 

— Eligibility codei equal to "G/' "K/* and "0" were 
excluded 

Records with blank name and address fields were excluded- 

There were several steps required in preparing the master 
Pell-Campus-Based and GSL-only sampling frame: 
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• Restrict tach univtrse file to tht 48 contiguous states, 

txclude schools in Alaska, Hawaii, and the Trust 
Territories 

• Delete all ineligible records, as defined below, on each 
of the three files (the definition of -ineligible*' was 
provided by the Department of Education) 

• Mer^i the 1985-86 Pell file with the 1984-85 file to gather 
any data on enrollment and count of recipients which were not 
available on the 1985-86 file 

• Reduce the Pell file to central offices and single-campus 
institutions since only these records have information on 
number of Pell recipients • In the case of multi-campus 
institutions, the central -office count of recipients 
represents the recipients at all the campuses 

• Create a master Pell-Campus-Based file, merging by entity 
identification numbers <EIN) 

e Delete ineligible records from the GSL file 

• Merge the master Pell-Campus-Based file with the GSL file, 
using the FICB which was present* on the GSL file and on the 
master file records coming from Pell 

• Delete cases on th^ master file that m^itcUed institutions 
participating in the Department of Education QC pilot study 

e Check for outlier cases in Pell and Campus-Based recipients, 
i.e., cases where the ntunber of recipienLs was unreasonably 
large when compared to the enrollment. 

After deleting ineligible records, the Pell file consisted of 5,337 
institutions. The corresponding Campus-Based file consisted of 4.430 
institutions. The GSL file consisted of 8,904 institutions. 



We encountered a number of problems in constructing the sampling 

frame for Stage Two. The necessary basis for merging the three universe 

files was a common identifier for each participating institution, which 

did not exist across all three files. For example, in merging the GSL 

universe file with the Pell -Campus -Based file we found that there was no 

coiwnon identifier across the Campus-Based and GSL files. Thus, 

Campus-Based-only (not offering Pell) institutions on the master file 
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could not be merged with the GSL file. Thus, the institution sample had 
to be constructed in two stages, a merging of the Pell and Campus-Based 
files and then a merging of this new file with the GSL file. 

The institutional identifiers - Entity Identification Numbers (EIN) 
for Pell and Campus-Based and FICE Codes for Pell and GSL - turned out 
not to be truly unique for each institution. For example, within the 
Pell and Campus-Based files, we found many cases where two different 
institutions had the same EIN. In cases where the institutions were 
branch campuses of the same institution, these could be resolved by 
cross-checking by zip code. In some cases there were two or more records 
with the same institution name and different EIN's; these were resolved 
by hand-checking. Across the Pell and Campus-Based files, we found that 
a number of institutions were- listed with the same name but different 
EIN's. Sometimes this was due to the fact that an institution may report 
at the branch level for one program and at the central office level for 
the other program. Again, these cases were hand-checked to resolve as 
many of these problems as possible. 

The corranon identifier between the Pell and GSL file is the FICE code 
(identification code of the central administrative office). Among the 
problems encountered with the FICE code were: 

♦ Blank FICE codes 

♦ FICE code was present but no institution name appeared on the GSL 
file 

♦ Duplicate FICE codes on the GSL file, but with different 
institution names 

♦ Different FICE codes for two records with the same Pell and GSL 
institution names. 



Any GSL record that did not merge by FICE code with the 
Pell-CampuS'Based file was assigned to the GSL-only franiG. Hcvever/ many 
of the^e records were, in fact* branches of Pell institutions that did 
not match by FICE with a record on the main Pell file or the file of Pell 
branches. In addition, the GSL universe file was not current. It 
included any institution that had ever certified a student for GSL. Once 
again, many of these cases were resolved through hand checking. 

The master file resulting from merging the Pell and Campus -Based 
files by EIN and deleting the Institutional Quality Control Pilot Study 
schools (41) consisted of 5,655 institutions. Out of a GSL file of 8,904 
institutions, 3,812 merged with the Pell-Campus-Based master file, 
resulting in an initial GSL-only frame of 5,092 institutions. 

3.2.2 Sttident Sanipling Frames 

The student sampling frame consisted of lists of Pell and 
Campus-Based recipients and of GSL certifications tor the 1985-86 
academic year. The most accurate r^d current lists existed only at the 
sampled institutions / and we took several steps to ensure both the 
availability and usefulness of the lists we needec 

When the Department of Education first notified the institutions that 
they had been selected for the study, the letter specified the 
information that would be requested during the scheduling call and at the 
time of the visit, including the sampling lists. When our telephone 
staff called the institutions to schedule the data collectors' visits, we 
asked them again to prepare the three lists: an up-to-date list of Pell 



Grant recipients in the 1985-86 academic year, an unduplicated list of 

actual *.eCx^xxsMU» wj. vwM^ua— fcjoo^G w^v* ^••^ ^xir «.^^^«.x..^ 4-. — ; ^ — s - 

list of GSL certifications made in 1985-86. The data collectors repeated 
this request when they confirmed their schedule a few days before the 
visit. 



Although most institutions did provide exactly the lists we ai;V0d 
for, some would not or could not (due to the lack of staff time or 
computer software). Therefore, data collectors in the field encoaacered 
a wide variety of "lists" which had to serve as sampling frames, 
including: 

• Separate lists for each program — Some schools provided 
separate lists of recipients in the Pell Grant program and in 
each of the Campus-Based programs. Students who received aid 
from more than one program would be on more than one list. 

• Lists of all aid recipients — Some institutions were 
able to provide only an unduplicated list of all aid 
recipients, with codes indicating what kinds of aid each 
student received. 

• Files of cards or folders — Some institutions had no 
list of Pell or Campus-Based or any other aid recipients. 
They had either a card-file index of all recipients or 
individual student file folders constituted their "list." 

Similar variety was encountered in "lists" of GSL certifications, which 
ranged from a sample list of all students certified to stacks of folders 
containing all the institution's copies of the current year's 
certification forms. 



We had encountered similar variety in lists of recipients in Stage 
One of the Title IV Quality Control Study. Therefore, we were able to 
train the data collectors in statistically valid ways of dealing with 
lists that d^^^^red from our ideal. Our field supervisor also provided 



technical advice on sampling problems to many collectors by telephone. 
The student sample sel^ctioa procedures iniplsiuoutod during ths 
institutional visits is described below in Section 3.4. 



3.3 SELECTION OF CTS INSTITUTION SAMPLE 



For reasons of sampling efficiency^ it was desirable to select the 
main institution sample/ i.e., the Pell-Campus-Based sampling frames, 
with probability proportional to a measure of size (MOS). The measure of 
size used in the institution sample selection was the number of Pell 
and/or Campus-Based aid recipients. We encountered several problems with 
measures of size. Information for some institutions on the 
Pell-Campus-Based file did not include the number of recipients and/or 
enrollment/ or had inaccurate recipient and/or enrollment information. 
In some cases, the count of Campus-Based recipients referred to the main 
campus while the count of P9ll recipients referred to all campuses. The 
GSL-only file did not contain any measures of size, either certifications 
or enrollment. 



If an institution's record did net have a value for count of 
recipients for either Pell or Campus-Based/ it was necessary to impute a 
value. The distribution of the 5,655 institutions on the master file by 
whether or not they had a value for count of recipients is shown in 
Exhibit 3-2: 





ON 
Ct FILE 
ONLY 


ON 
PELL PILE 
ONLY 


ON 
PELL AND 
CB FILES 


1 

TOTAL 


MOS AVAtUBlH 


344 


ij4a 


3.028 


5.421 


MOSNOTAVAIU8LE 


40 


161 


25 


234 


TOTAL 


362 


1,310 


3,953 


5.655 
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A regression model was used to impute the missing measure of size. 
In addition to the institution's control (private. public, or 
proprietary) and the type of program offered (2-year/ B.A./B.Sw 
graduate), enrollment was the only other appropriate independent variable 
available for most cases with missing count of ri^cipients for both 
programs. The value, which is a measure of the proportion of 

variation explained by the independent variables (enrollment and type and 
control), obtained from the Pell regression was .58 and for the CB 
regression it was .57. These R values indicate a reasonably good fit?^'^<*y 

Where the information necessary to impute a MOS was missing or for 
institutions for which the predicted number of recipients was less than 
six recipients, a minimum of six recipients was assigned. This ensured 
that students within that school would be selected at a rate that would 
preserve the self -weighting characteristic of the student sample. 

3.3.1 Computation of the Measure of Size 

The sample design for the institution sample had to take into account 
the fact that separate student samples, with specific sample sizes for 
each program, would be drawn at the institutions. Thus, an important 
design issue was how to assign a measure of size to an institution so as 
obtain a reasonably efficient institution sample for selecting separate 
student samples for the three programs. The approach we followed was to 
assign to an institution ^ measure of size equal to the maximum of the 
proportion that the Pell and Campus-Based recipients at the institution 
represented of the total Pell and Campus-Based recipients, respectively. 
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That is: 

Mi - max of ^ij 

(over j) N^j 

where 

i s identifies the institution 

j identifies the program (Pell or CB) 

Mi - the institution measure of size used in the sample 

selection for the i-th institution 

Nij s the number of recipients at the i-th Institution 

in the j-th program 

NJ - the number of recipients in the j-th program over 

all institutions. 

As indicated earlier, the GSL file had no information on the number 
of GSL certifications at an institution. Thus, an institution's measure 
of size was based on information on recipients for Pell and Campus-Based 
only, even if it had been identified as also having GSL certifications. 
However, we guarded against the impact on the weighting of a situation 
where an institution with just a few Campus-Based and Pell recipients 
(and thus a large weight) turned out to have a large number of GSL 
certifications. We doubled the measure of size whenever the proportion 
that an institution's enrollment represented of the total enrollment of 
GSL institutions was greater than two times the measure of size of that 
institution. 



3.3*2 Deteroiination of Certainty Institutions 



A sample of 290 institutions was to be selected from the Pell-Campus- 
Based master file. Institutions with a measure of size greater than the 
overall selection interval (total measure of size/290) were drawn into 



I. - 



CRiC 3-12. 7 



the sample with certainty* The certainty cutoff was set at: 



Total measure of size 

Institution sample size 



1,43166 
290 



• 07, 



Eight institutions were drawn into the sample through this process. 
3.3.3 Selection of Noncertainliy Institutions 

The remaining 282 sample institutions were selected from those 
remaining in the frame after the certainties were removed. The sample 
design for this noncertainty portion of the sample was basically a double 
sampling with probability proportional to size (PPS) selection of 
clusters and PPS selection of individual institutions from those in the 
sample clusters. The sample design called for: 

• Ordering the file by geographic code and forming clusters of 
consecutive schools of a minimum of eight schools each 

• Sampling clusters ' with probability proportional to the 
measure of size of the cluster 

• Assigning a weighted measure of size. WMjj^ to schools 
within sampled clusters where 



WMij a Mij/Pj 



i 



identifies the institutions 



j 



identifies the cluster 



Mij 



measure of size of the i-th institution 
in the j-th cluster 



3 probability of selection of the j-th cluster 



Samplirg institutions systematically from the file in cluster 
order, with probability proportional to the weighted measure 
pf size (WMij). 
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ZIP Recede* The frame of institutions for the selection of the 
noncertaincy portion of the ^aiftple was first merged with a 2I?-cods- 
recode file. The ZIP-code-recode ^tep attached a serpentine geographic 
code to each institution and removed invalid three-digit ZIP'S (those 
that did not match the master list) for verification and correction, 

Fonnation of Clusters. To form geographic clusters of 
institutions, we sorted the file first by the geographically contiguous 
sort variable (ZIP-recode) and within that by 5-digit ZIP code. The rule 
used in forming clusters was that: 

• Each cluster must contain a minimum of eight consecutive 
institutions 

e All schools within the same 5-digit ZIP code must be in 
the same cluster. 

The clustering procedure resulted in 655 clusters with an average of 
8.7 schools per cluster. 

Sample of Clusters. Out of the 655 clusters in the frame, we 
sampled 100 clusters, with probability proportional to size (PPS). The 
measure of size of a cluster was the sum of the measure of size of the 
institutions in the cluster. The 100 sampled clusters contained 878 
schools, and none of the cluster sises exceeded the sampling interval, 
which would have required their selection with certainty at this stage. 

Before drawing th« institution sample an effort was made to eliminate 
from the sample clusters any duplicate institutions; that is, 
institutions that were on the Pell and on the Campus-Based universe files 
but because they had a different EIN number on each file they did not 
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merge in the computer operation. 



Sampling Institutions. In the second stage of sampling the measure 
of size for the PPS selection was the institution's MOS weighted by the 
cluster weight (the reciprocal of the cluster's probability of 
selection); that is. 



WM. . = M. ./P. 

The institutions in the 100 sample clusters were retained in cluster 
order and a systematic sample of 282 institutions was selected with 
probability proportional to the weighted measure of size (WM^^). A 
considerable number of institutions (108) had a measure of size larger 
than the sampling interval and were conditional certainties at this 
stage; these were removed from the frame before the remaining 174 
noncertainty institutions were drawn. After looking closely at the 
institution sample it was determined that two of the sample institutions 
were in fact the same. This reduced the actual sample from the 
Pell-Campus-Based master file to 289 institutions. 



3. 3^4 Distribution of the Saraple Among Type-Control Strata 

In Stage One, at the second stage of sampling, institutions were 
stratified by institution control and type (length) of program. ThuS/ it 
was reasonable to expect that the sample estimate of recipients, by 
control/ length stratum, should be close to proportional to the 
distribution of the measure of size by stratum. In Stage Two, however, 
because the frame was a combined frame of Pell, Campus-Based, and GSL 
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institutions, the institution sampling frame was not stratified by 
institution control and type of program • ThuS/ vhsa comparing the 
distribution of the Pell or Campus-Based sample to the corresponding 
universe, the distributions are not expected to be proportional. Exhibit 
3-3 shows the distribution of universe and sample institutions by 
control/ length stratum for the Pell and Campus-3ased programs. Exhibit 
3-4 shows the distribution of the universe counts and sample estimates of 
recipients for Pell and Campus-Based, by institution control/length 
stratum. There is no comparable distributional data for GSL, since the 
GSL universe file did not contain MOS information. 

3 .3 .5 Subsampling Branches 

For some institutions, the count of recipients on the Pell and 
Campus-Based universe files included recipients at the main campus and 
all the branches; however, the student aid records were kept at the 
individual campuses. The universe file indicated this situation and a 
list of branches was included on the file, but without a count of 
recipients at the individual campuses. For these institutions/ we 
obtained enrollment figures for the branches from the "Education 
Directory of Colleges and Universities 1983-84," NCES, and subsampled one 
branch from each institution with probability proportional to enrollment. 

During scheduling calls to sample institutions, we uncovered 
additional institutions where the student aid records were at the branch 
campuses. We obtained enrollment figures for the branches and followed 
the subsampling procedure described above. A total of 48 institutions 
were subsampled. 
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1985 « 86 
ikieTi-riiYtnMC 



< 2 YEARS 
NUMBER % 



2 . ■< 4 YEARS 
NUMBER % 



4 * YEARS 
NUMBER % 



TOTAL 
NUMBER % 



PELL 

UNIVERSE 
Public 
Private 
Proprietary^ 

TOTAL 



SAMPLE 
Public 
Private 
Proprietary^ 

TOTAL 



114 


2.2 


941 


17.9 


460 


8.7 


1.515 


28.8 


48 


0.9 


368 


8.0 


1.140 


21.7 


1,556 


29.6 


1,818 


34.5 


349 


6.6 


25 


0.5 


2,192 


41 .6 


1,980 


37.6 


1,658 


31.5 


'.625 


30.9 


5,263 


100.0 


2 


0.7 


73 


26.4 


73 


26.4 


148 


53.4 


0 


0.0 


7 


2.5 


84 


30.3 


91 


32.9 


25 


9.0 


10 


3.6 


3 


1.1 


38 


13.7 


27 


9.7 


90 


32.5 


160 


57.8 


277 


100.0 



CAMPUS-BASED 

UNIVERSE 
Public 
Private 
Proprietary^ 

TOTAL 



SAMPLE 
Public 
Private 
Proprietary^ 

TOTAL 
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32 


0.7 


801 


18.4 


488 


11.2 


1,321 


30.4 


33 


0.8 


246 


5.7 


1,194 


27.5 


1,473 


33.9 


1.245 


28.7 


280 


6.4 


26 


0.6 


1,551 


35.7 


1,310 


30.1 


1,327 


30.5 


1,708 


39.3 


4,345 


100.0 


0 


0.0 


67 


24.9 


75 


27.9 


142 


52.8 


0 


0.0 


7 


2.6 


87 


32.3 


94 


34.9 


23 


8.6 


7 


2.6 


3 


1.1 


33 


12.3 


23 


8.8 


81 


30.1 


165 


61.3 


269 


100.0 




EXHIBIT 3-3 

DISTRIBUTION OF TITLE IV INSTITUTIONAL UNIVERSE AND SAMPLE 
FOR THE PELL AND CAMPUS-BASED PROGRAMS 
BY INSTITUTION TYPE AND CONTROL 
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1985 - 86 
AID RECIPIENTS 





< 2 YEARS 


2 . < 4 YEARS 


4 ♦ YEARS 


TOTAL 




NUMBER 


% 


NUMBER 


% 


NUMBER 




NUMBER 


% 


PELL 


















UNIVERSE 
Public 
Private 
Proprietaiy^ 


19,134 
5.126 
305.193 


22 
0.9 
34.5 


633.586 
45.249 
136.250 


26.0 
1.9 
5.6 


892.408 
382.628 
15.485 


36.6 
15.7 
0.6 


1.545.128 
433.003 
456.928 


63.5 
17.8 
17.8 


TOTAL 


329.453 


37.6 


815.085 


33.5 


1.290.521 


53.0 


2.435.059 


100.0 


SAMPLE 
Public 
Private 
Proprietary 


13.739 
0 

205.281 


0.6 
0.0 
8.9 


629.688 
54.898 
113.501 


27.1 
2.4 
4.9 


880.371 
380.618 
35.135 


38.1 
16.5 
1.5 


1 .520.798 
435.516 
353.917 


65.8 
18.9 
15.3 


TOTAL 


219.020 


9.5 


795.087 


34.4 


1.296.124 


56.1 


2.310.231 


100.0 



CAMPUS-BASED 


















UNIVERSE 
Public 
Private , 
Proprietary 




2.550 
1.335 
78.142 


0.2 
0.1 
5.8 


179.351 
23.887 
32.732 


13.2 
1.8 
2.4 


571.520 
460.149 
8.348 


42.1 
33.9 
0.6 


753.421 
485.371 
119.222 


55.5 
35.7 
15.3 


TOTAL 




82.027 


95 


235.970 


17.4 


l.f-^.017 


76.6 


1.358.014 


100.0 


SAMPLE 
Public 
Private ^ 
Proprietary 




0 
0 

43.354 


0.0 
0.0 
32 


202.827 
26.252 
15.389 


14.9 
1.9 
1.1 


559.547 
503.414 
12.549 


41.0 
36.9 
0.9 


762.374 
529.666 
71.292 


55.9 
35.7 
8.8 


TOTAL 




43.354 


3.2 


244.468 


17.9 


1.075.510 


78.9 


1 .363.332 


100.0 
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EXHIBIT 3-4 

DISTRIBUTION OF TITLE IV UNIVERSE RECIPIENTS AND SAMPLE 
ESTIMATE OF RECIPIENTS, FOR THE PELL AND CAMPUS-BASED 
PROGRAMS BY INSTITUTION TYPE AND CONTROL 
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3.3«6 The GSL-Only Institution Sample 



Travel considerations made it desirable to restrict the GSL-only 
sample to the same 100 geographic clusters that comprised the 
Pell-Campus-Based sampling frame. An algorithm was developed to extract 
from the GSL file ail schools which fell in any of the 100 selected 
clusters on the basis of the geocode of the cluster and the five-digit 
ZIP Code. This reduced the effective frame of GSL schools to 743 
institutions. 

Out of the 878 institutions appearing on the Pell-Campus-Based 
100-cluster frame, 603 had been matched to the GSL frame through the FICE 
code. A listinq of the 878 institutions was produced, sorted by cluster, 
and alphabetically within cluster. A list of the 743 GSL-only schools, 
sorted in the same way, was also produced. Clerks compared the 
100-cluster Pell-Campus-Based list to the lOO-cluster GSL-only list. Any 
institution appearing on both lists was removed from the GSL-only frame. 
A strict rule was used in deciding what was to be considered a match 
between the two lists: since the Pell file defined an institution as an 
independent campus or a central office on the basis of the institution's 
reporting procedure rather than whether it was in fact a branch campus, a 
central office, or an independent institution, we did not delete from the 
GSL-only file any institutions that in fact had no other chance of coming 
into the sample. This procedure yielded a final GSL-only frame of 489 
institutions. 

Because of the poor quality of the frame, we sampled 50 GSL-only 
records in order to guarantee an actual yield of 25. First, the list of 
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489 GSL-only schools was sorted by cluster number. A PPS sainple of 50 
schools was selected using the cluster selection weight as the measure of 
size. This resulted in a clustered, equal probability sample of GSL-only 
schools . 

After the sample of GSL-only institutions was drawn, a very U^.orough 
check was made to determine if these institutions had another chance of 
coming into the sample through the Pell-Campus-Based file. This check 
relied on the Pell-Campus-Based master file, lists of branch campuses, 
and telephone calls to the sample schools when necessary. It reduced the 
actual GSL-only sample to 36 institutions. 

Because of limitations on the budget and on the time required to 

visit sample institutions, a random subsample of 25 GSL-only insti^-.utions 

was selected from the 36 identified as eligible through the process 
described above. 

3,3.7 Institution Weights 

In general, the overall weight associated with an institution in the 
Stage Ivo sample was the product of the cluster weights and the within- 
cluster institution 'eight. However, as described earlier, when student 
aid records were at the branch campuses, but the sample unit was the 
institution as a whole^ we subsampled a branch for the field visit. As a 
" ilt of the subsampling, the overall weight for these institutions 
includes a subsampling component in addition to the cluster sampling 
component and the insti tution-within-cluster component. Thus, the 



overall institution weight, W., may be expressed as 



Wi 



WCi Wli WBi 



wh©re 



WCi 
Wli 
WBi 



the reciprocal of the cluster selection 
probability 

the reciprocal of the institution selection 
probability conditional on the cluster 

the reciprocal of the branch subsainpling 
probability. 



3.4 SELECTION OF THE STUDENT SAMPLE 

The Stage Two student sample was designed to result, in a fixed 
overall sampling rate for students in each of the three programs. Th'is^ 
the within-school student sampling rate for a particular program was a 
function of the overall sampling rate desired for that prograir. and the 
institution base weight (the reciprocal of the institution's overall 
selection probability). That is. 



where 



^si 

fsi 
f 

W; 



f* Wi 



Student sampling rate within institution i 

overall desired sampling rate for the specified 
program 

overall institution weight (as defined in 3.3.7 
above ) . 



The desired overall sainpling ra\.es, t, for the three programs are given 
below; 



Pell: 

CBt 

6SL; 



f3 



1,300/2,436,480 

1,511/1,358,014 

400/324,700 
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fit 



tihere the numerator is the target student sample size specified for the 
program/ and the denominator is the best sstiniatc available of the total 
number of students in the program. 

Thus, the witjiin-school sampling rates are given by: 



Pell: 


fsil = 




Wi 


CB: 


fsi2 = 




Wi 


GSL: 


fsi3 







Then, the expected student sample size for Pell and Campus-Based in 
institution i is given by: 

Pell: nil = fsil 
CB: ni2 = fsi2 ^12 



where 



the expected sample size in institution i, program j 



Nij = the expected number of program recipients at the insti- 
tuition i/ program j. 

Thus the initial caseload/ and hence the amount of time scheduled for 
the data collector's visit to each institution/ was based on two 
assumptions: 

• That the number of Pell and Campus-Based recipients in 
1985-86 would be proportional to the 1984-85 figures 

• That the relatively low sampling fraction es wished for GSL 
would result in a small number, generally less than four, of 
GSL students sampled. 

3.4.1 Drawing the Student Sample 

As part of the telephone calls to schedule the data collector's 

r-n^i^visit/ the financial aid offices were asked for current estimates of Pell 
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and Campus-Based recipients and GSL certifications. Based on these 
2>evised measures of size, we recomputed the within-institution sampling 
rates and estimated caseload. In some cases the resulting caseload for 
an institution was larger than could be accommodated in the scheduled 
time for the visit. For these institutions the sampling rate was cut in 
half. 

After the revisions described above, sampling worksheets for each 
institution and program were produced. As shown in Exhibit 3-5, the 
worksheet provided a check jn whether the number of recipients on 
the sampling li'jt was within, the range we expected based on prior 
information. It also summarized the steps to be followed for sampling, 
identlfieia the sample line numbers for each program, and provided a range 
check for the resulting sample. 



At the institutions, data collectors selected sample students from 
the sampling lists (see Section 3.2.2) following the procedure specified 
on the sampling worksheet. The sampling worksheet allowed for the actual 
number of recipients (thus the sample size or "take") for each program to 
vary as much as 50 percent from the expected number of recipients. 
Within that range, the data collector drew the sample as dictated by the 
sample line numbers on the worksheet. However, if the actual number of 
recipients fell outside of the range, the data collector called the field 
supervisor. We had developed procedures for adjusting the sampling rate 
accorcling to the actual number of recipients in the program. The 
sampling rates were modified to produce caseloads consistent with the 
amount of time scheduled for the visits and new line numbers were 
generated for sample selection. The correct number of recipients and the 



SAMPLING WORKSHEET 
01-03 MCINTOSH COLLEGE 
DOVEf^NH 

A. PELL PROGRAM 

SAMPLING STEP3: 

A1. NUMBER CONSECUTIVELY ALL NAMES ON THE UST. 

A2. IF YOU HAVE A CLEAN UST (NO INEUGBLE NAMES), COMPARE THE NUMBER OF PELL RECIPENTS ON THE UST TO THE 
MINIMUM AND MAXIMUM NUMBERS BELOW. 

MINIMUM: 50 MAXIMUM: 150 

IF FEWER THAN MINIMUM OR MORE THAN MAXIMUM. CALL BETH SCHWARtZ (SPRINT) 627-2914. 



A3. USING THE LINE NUMBERS LISTED BELOW, IDENTIFY THE SELECTED STUDENTS BY CIRCLING THE SAMPLE LINE NUMBERS. 
SELECT: 



11 

327 
643 



32 
348 
664 



54 

369 
685 



76 
390 
706 



96 
411 

727 



117 
432 
749 



138 
453 
769 



159 
474 

790 



180 
496 
811 



201 
517 
832 



222 
528 
853 



243 
559 
853 



264 

510 
895 



285 306! 
601 622 \ 
916 928 1 



A4. COMPARE THE NUMBER OF SAMPLE RECIPIENTS TO THE MINIMUM AND MAXIMUM NL **'=^ERS BELOW: 

MINIMUM: 2 MAXIMUM: 8 

IF FEWER. CAa BETH SCHWARTZ (SPRINT) 627-2914 
IF MORE, CALL BETH SCHWARTZ (SPRINT) 627-2914 

A5. NOTE: IF NO STUDENTS SAMPLED, USING THE LINE NUMBERS IN A3 COMPUTE: 



.5151 X 



RECORD THE NUMBER OF 
STUDENTS ON YOUR LIST 



SELECT FIRST LINE NUMBER GREATER 
THAN OR EQUAL TO THIS NUMBER 



A6. NOTE: IF THE LIST OF NAMES IS NOT CLEAN (INCLUDES INELIGIBLE NAMES), AND THE LAST SEQUENCE DUMBER ON 

YOUR LIST (ENTER: ) IS GREATER THAN THE LAST Li^^E NO. IN A3: GENERATE ADDITIONAL LINE 

NUMBERS BY ADDING 2t TO THE LAST LINE NO. ON A3. CONTINUE GENERATING LINE NUMBERS UNTIL YOU 
REACH A NUMBER GREATER THAN THE UST SEQUENCE NUMBER ON YOU LIST. 




EXHIBiT 3-5 
EXAMPLE OF SAMPLING WORKSHEET 
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revised within-institution sampling rates were recorded by the Advanced 
Technology field supervisor to allow adjusting the sampling weight as 
required* 

In some instances , even if the number of recipients was within the 
acceptable range, the actual take exceeded the time allowed for the 
visit. If the data collector could adjust his/her travel schedule to 
extend the visit, adjusting the take was not necessary. If not, the 
sampling rate was cut in half by selecting every other recipient to yield 
an acceptable sample size. 

Midway through the field period, our field receipt control system 
revealed that for a number of institutions the actual takes were within 
the allowable range, but below t. axpected figures. This meant the 
sample was falling behind the projected size. For the second half of the 
field period, we adjusted the sampling rate to increase the take in those 
institutions where the schedule permitted it in order to maintain the 
required total student sample size. One effect of this procedure, 
however, was the introduction of more variability among the sampling 
weights with a resulting loss of efficiency in the sample estimates. 
This is described in more detail in Appendix A. 

3.5 SESPONSE SATES 

Response rates for Stage Two were satisfactory. As v;ill be discussed 
in detail below, we visited 297 institutions out of a sample of 314. All 
eligible institutions were visited (functioning, locatable institutions 
participating in one or more of the Title IV programs). The total number 
of students actually sampled was 2,996 or 93 percent of the projected 



number • We experienced a sample loss of 14 cases, which yielded a 
scudenc sample of 2/932 cas&s, or 92 • 3 psrcsnw of tho prcjoctcd figure. 
These cases were sampled in error since all had received only a PLUS Loan 
and were therefore not in the sampling frame. The distribution of 
sampled cases by type of program is shown in Exhibit 3-6. The interview 
response rate for all sampled students and parents was 87 ,3 percent/ 
which was an increase over Stage One response rates of slightly less than 
80 percent. This rate was achieved in spite of the fact that 
participation in the interviews was voluntary for recipients of 
Cojnpus-Based aid and GSL*s. as was also the case in Stage One for all 
sampled stvdents. 

3.5.1 Institutional Data 

The target size of the sample was 300 inscicutionS/ including 25 
GSL-only schools. The original institutional sample consisted of 339 
institutions: 289 institutions drawn from the Pell and Campus-Based 
universe files and 50 institutions drawn from the GSL file. (The initial 
GSL sample was 60 percent larger than the corresponding Stage One sample 
because of a sample loss of over 50 percent in Stage One GSL-only 
institutions. ) 

In a preliminary attempt to "clean" the GSL-only sample/ we 
eliminated 14 institutions as being ineligible for the sampling frame/ 
i.e./ these institutions were discovered not to be GSL-only 
participants. A second round of deselection/ based on equal probability/ 
eliminated another 11 institutions/ resulting in the target GSL sample 
size of 25/ and a total sample of 314 institutions. 
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Three institutions were deselected from the Pell-Campus-£ased list. 
These institutions all use the services of a particular consultant to 
manage their financial aid programs. This consultant manages a large 
number of proprietary schools, such that two or three of them have been 
drawn in each of our QC study samples. These institutions were 
deselected from Stage Two on the basis of a history of nonparticipation 
in previous QC studies. 

We therefore begc.n scheduling calls with a sample of 311 
institutions. In spite of attempts by both Westat and ED staff to locate 
all institutions, five institutions were discovered to be out of business 
and four more were non-locatable (these were all from the GSL-only 
sample). In addition, one institution proved to be very difficult to 
locate. Once located, repeated attempts to contact the director to 
schedule a visit were unsuccessful. One other sample institution was 
discovered to be a branch of another sample institution. It was 
therefore dropped from the sample and the branch campus subsampling 
procedure was used to determine the appropriate campus to visit. 

One institution, part of a chain of proprietary schools, was 
discovered to have branch campuses and was subsampled. Since the records 
for all the schools in the chain were located in one place, with one 
financial aid director, the site visits were combined to yield separate 
student samples, but one interview. 



All 10 institutions which initially refused to participate in the 
study eventually agreed to be visited, although this did not happen for 
?11 of these cases before the start of the field period. One institution 
did not agree to participate until the fourth week of data collection. 
Since this study was an official inquiry of the U.S. Department of 
Education, for which Advanced Technology was acting as agent, schools 
participating in Federal aid programs did not have a legal right to 
refuse. At two institutions, however, the FAA did not grant an interview 
to our data collector (the interview segment of the visit was 
voluntary). In both cases the data collectors were able to obtain the 
information they needed to complete the Student Record Abstracts from 
other sources in the financial aid office, and to provide u^ with enough 
information on institutional policies to enable us to analyze the sample 
cases . 

The final institutional sample consisced of 297 institutions. This 
represented 283 institutions drawn from the Pell-Campus-Based sample, or 
98 percent/ and 14 institutions from the GSL sample of 25, or 5G percent, 
for a total participation rate of 94.6 percent (297 of 314 institutions 
in the sample). 
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3.5.2 -Student and Parent Data 



Response rr^tes for student and parent interviews were continuously 
monitored during the field period through the Automated Survey^ Control 
System (ASCS). At the conclusion of the field period, the response 
statistics generated by ASCS were reconciled with those keyed into the 
receipt control file as each of the questionnaires was logged into the 
receipt control system at Westat. Response rates were tracked using a 
set of status codes, which indicated the final disposition of each of the 
sampled cases. The disposition codes are shown in Exhibit 3-7; final 
disposition of all sampled student and parent cases is shown in Exhibit 
3-8. Response rate statistics were monitored for each student and parent 
group (e.g.. all students, independent students, dependent students, all 



CODE 

11 
12 

13 
14 

15 
16 
17 
18 
20 
21 
22 



OERNITION 



Completed lnt«rvf«w with usable data 

Respondent not at home after a minimum of five In- 
person cals by the interviewer 

Respondent cannot be located 

Respondent has an extended Illness and is unavailable 
during the field period 

Refusal or break-off 

Avoid lnte(view 

Language problem, no interpreter available 
Other 

Respondent was sampled In error 

Respondent Is out of the country during field period 

Respondent is deceased 



1 
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DISPOSITION 



RNAL DtSPOSmONSt ELIGIBLE STUDENTS 
NUMBER 



PERCENT 



Compl9t« (11) 

Not Hom«, Maximum Calls (12) 
Can1Locat8(13) 
Extended Ulness (14) 
R'^fusal/Breakoff (15) 
Avoider (16) 
Language Barrier (17) 
Other (18) 
Out of Country (21) 



2,629 
29 
85 
6 
155 
38 
1 

24 

16 



2*983* 



88.13 

0.97 

2.85 

0.20 

5.20 

1.28 

0.03 

0.80 

0.54 
100.0 



* 13 09M$ W0r0 CkJt-ofScop^ ($mpilng mror, d$cm»ed), Mnglny the 
totBi to 2,996 (2,9$3 -h 13). Th0$0 caset are exduded from th0 calculation 
ofnsponsa rates arid Mis on^ttad from tha tabia. 



EXHIBIT 3*8 FINAL DiSPOSlTION OF CASES 



RNAL DISPOSITIONS, ELIGIBLE PARENTS 
DISP03ITI0N NUMBER PERCENT 


Complete (11) 


2.391 


85.55 


Not Hornet Maximum Cafis (12) 


25 


0.89 


Cani Locate (13) 


80 


2.86 


Extended Illness (14) 


17 


0.61 


Refusal/Braakcff (15) 


217 


7.76 


Avoid«r (16) 


22 


0.79 


Language Barrier (17) 


11 


0.39 


Other (18) 


32 


1.15 




2,795 


100.0 
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parents, independent parents, dependent parents, independent students and 
parents, dependent students and parents , and student/parent pairs). 
Response rates by student dependency status are shown in Exhibit 3-9» 



RESPONDENT 
GROUP 


COMPLETED 
INTERVIEWS 


RESPONSE 
RATE (%) 




Dependent 
Student 
Parent 


1.5M 
1.531 


89.8 
87.0 




Independent 
Student 
Parent 


1.041 
860 


87.8 
83.1 


ran 



EXHIBIT 3-9 RESPONSE RATES BY DEPENDENCY STATUS 

Response rates were calculated as the percent of questionnaires that 
were completed of the total number of sampled students and/or par<*nts for 
whom an interview was possible. If a sampled student or parent, for 
example, was out of the country during the field period, or was deceased, 
they were not counted in the calculation of the response rate because an 
interview with them was not possible. However, sampled students or 
parents who could not be found, or who refused to be interviewed were 
included in the calculation of response rates as nonrcspondents . 

The total number of students and parents for whom an interview was 
possible was 5,778. Completed interviews were obtained for 5,020, for a 
response rate of 86.9 percent. For students, only 2,629 interviews were 
completed out of 2,983 possible interviews, for a student response rate 
of 88.8 percent. For parents only, there were 2,795 parents for whom an 
interview was possible. There were 2,391 completed parent interviews, 
for a response rate of 85*5 percent. 



In addition, there were 2/262 "pairs" of interviews completed, where 
both student and parent questionnaires were obtained. 0£ these pairs / 
1,464 were dependent student/parent pairs, and 798 were independent 
student/parent pairs. 
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INSTITUTIONAL DATA COLLECTION 



The institutional data collection constituted the first, critical 
phase of the study. The lack of comprehensive Federal recipient data 
bases mandated that the institutions be the resource for identifying 
complete l^sts of current recipients from which a sample could be drawn. 
In addition, the institutions are a legitimate focus of the study in 
their own right, in that institutional error is a component of 
program-wide error. Finally, information found in a student's 
institutional file provided us with an additional source of student data. 

In this chapter we describe the institutional data collection 
conducted by Advanced Technology in February and March 1986. Information 
on 2,996 sampled students was obtained during visits to 29^ 
institutions. We will describe how our field staff was recruited and 
trained, how the institutional visits were scheduled/ supervision of the 
visits, and our procedures for handling the data collection materials as 
they were received from the field. 

4.1 PROJECT STAFF 

The tasks involved in the institutional data collection required a 
variety of skills and experience in student financial aid, quality 
control, program evaluation, and survey research. Advanced 
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Technology's project staff are experienced in tfeese areas, and have a 
clear understanding of the process and requirements not only of gathering 
the data, but also of preparing^ processing, and interpreting that data. 
Our project manager and task managers brought to Stage Two specific 
experience from both Stage One and the Pell QC studies. 

4.2 FIELD STAFF 

Given the complexity of the Title IV programs and the variety of 
documents and institutional recordkeeping systems we were likely to 
encounter, strong, current financial aid experience was the most 
important qualification for the da^a collectors. Since the data 
collectors were working alone all over the country, constant supervision 
of each one was impossible; therefore, they had to be reliable and 
experienced enough to be able to make their own decisions in many 
situations. 

4 • 2 • 1 Trainer/Monitors 

Three data collectors who had demonstrated exceptional capability 
during Stage One were recruited to assist our project staff during data 
collector training, and to monitor the data collectors during the first 2 
weeks of the field period. Their first--hand experience in QC data 
collection as well as their expertise in financial aid made them a most 
valuable and efficiant resource ^oj us. Two ot them continued as data 
collectors once their monitoring responsibilities were completed. Their 
professional qualifications are included with those of the rest of the 
field staff below. 
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4.2.2 Data Collectors 



We began our recruitment of data collectors from the individuals who 
had performed successfully on the data collection for Stage One of the 
Title IV QC project^ which occurred in the spring of 1984. Nine of these 
experienced data collectors were available and were rehired. 

We advertised our field staff needs in the Newsletter of the National 
Association of Student Financial Aid Administrators (NASFAA) and the 
Chronicle of Higher Education . The NASFAA Newsletter provided th-i 
greatest number of new applicants with the type of experience we were 
seeking; most of the new data collectors we hired had applied in response 
to this advertisement. While we received several applicants in response 
to our advertisement in the Chronicle of Higher Education , the majority 
of these were well-qualified in higher education and student services in 
general^ but were lacking the kind of experience in financial aid which 
we required. Our other major source of appropriate candidates was the 
network of professional relationships and contacts within the financial 
aid community which we have developed through previous data collections, 
and which we were able to draw upon successfully once again. 

These sources produced a group of candidates from across the United 
States. This was desirable because it permitted us to hire many people 
to work in or near their home areas. They were familiar with regional or 
local peculiarities, such as procedures or records required by state 
agencies or policy recommendations of the state or r^^gional association 
of financial administrators. They would also have to travel shorter 
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distances and could often return home on weekends. This both reduced our 
travel expenses and maintained the moralo of the data collectors. While 
we were not able to assign everyone to a region close to home, we were 
able to make these kinds of assignments for the majority of field staff. 

The geographic dispersal of the applicants prevented face-to-face 
interviews but, after initial screening of resxunes, the most promising 
candidates were interviewed by telephone. We then made telephone checks 
of three references for each candidate before making an offer. The final 
field staff of 29 people consisted of 3 trainer/monitors, 22 full-time 
data collectors, and 4 alternates. 

The field staff had over 300 combined years of professional financial 
aid experience, averaging nearly 11 years each. Of the 29, 20 had 10 
years of experience or more; 5 had at least 15 years. Eighteen held 
positions at the level of financial aid director, with responsibility for 
the full range of financial aid activities (including a few individuals 
who had been directors at more than one institution). Six more had 
experience at the ^associate or assistant direccor level. The 
institutions represented by the data collectors included all types and 
controls. The educational backgrounds of the data collectors included 3 
with an earned doctorate, 14 with master's degrees, and 10 with 
bachelor's degrees. Several had more than one master's or additional 
course work beyond a master's degree. Virtually all the data collectors 
were affiliated with their state and regional professional associations, 
and with NASFAA. Many had participated in leadership roles at 
conferences and training programs sponsored by these associations, as 
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well as those sponsored by the U.S. Department of Education. Several had 
also served as consultants to their state governments in matters relating 
to postsecondary education in general and financial aid in particular. 

The field staff were well-equipped by training and experience to deal 
with the variety of documents and record systems they would encounter in 
the field/ to explain interview questions to financial aid directors/ and 
to understand answers couched in financial aid terminology. They could 
find relevant data in student files quickly and accurately. Our training 
program concentrated on ensuring consistent use of the data collection 
instruments/ with no need for instruction on the basics of the Title IV 
programs . 

4.3 SCHEDULING 

The primary objective of the scheduling plan was to allow 25 data 
collectors to travel as inexpensively as possible while visiting all the 
sample institutions within the 6-week data collection period. We 
developed an ideal master schedule to achieve this goal and then adjusted 
it as necessary when we could not visit specific institutions on our 
preferred dates. 

4.3.1 Master Schedule 

Our first step in constructing the ideal schedule was to plot the 
institutions to be visited on a map. Our objective was to divide the 
institutional sample as a whole into 25 regions/ and then to break each 
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region into l-week clusters. By first looking at the sample 
geographically, we saw that a fairly even distribution was possible. 
This gave us an important measure of flexibility in choosing dates for 
visits, allowing us to plan according to the logistical implications of 
our choices. 

Once we determined our data collectors' destinations, we then 
estimated how much time was needed tor each institutional visit. Time 
was required for four things: traveling to the school; interviewing the 
financial aid administrator (FAA); selecting the sample; and abstracting 
data from the files. We estimated how much time would be required for 
each task, and then estimated how long each visit was li^aly to take. 
However, since our estimates were based on information gathered during 
the scheduling calls, we had to allow for circumstances where actual 
caseloads would turn out to be higher than our estimates. We also had to 
allow for the possibility of extra travel time, in case our estimates 
turned out not to reflect local road conditions, weather, etc. Finally, 
we tried to overestimate the expected workload to allow for maximum 
flexibility in schedule adjustments during the scheduling and/or field 
periods. 

We were also able to use what we knew about previously visited 
institutions, in terms of the organization of the financial aid office 
and student files, to adapt the scheduled length of the visit to 

^ Although we had 25 regions, we used 29 individuals as data collectors 
becauso some were not available fOL' the full field period, a few 
remote institutions did not fit conveniently into any region, and 
substitutes would be needed in cases where illness or bad weather 
interfered with travel. 
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accommodate expected sampling problems. This knowledge was ^ery helpful 
in the case of one particular institution, where we knew that the data 
collector would have to sample directly from filing cabinets, unsorted by 
enrollment and type of aid, and at another institution where the data 
collector would be sampling from file cards. 

Once institutions had been grouped into regions, and the number of 
days needed for each visit had been estimated, we began assigning 
preferred dates to the institutions. We tried to designate at least one 
alternative date for each school, although this was not always 
practical. Where we had several schools to visit in a single city or 
within a few miles of each other, we could easily shift the preferred 
dates. In other areas, a region could be treated as a loop itself, with 
the preferred schedule specifying travel in one direction and an 
alternative schedule providing for travel in the reverse direction. The 
final master schedule consisted of a list of 311 participating 
institutions with preferred dates and alternative dates for each over a 
6-veek period. 

4.3.2 Setting the Actual Schedule 

As soon as the institution sample was drawn, the Office of 
Postsecondary Education (OPE) sent letters to the presidents and 
financial aid administrators of all sampled institutions, notifying them 
that th. ir institutions had been selected and requesting their 
cooperation, and inform^ung them that they would be called to arrange a 
date for the visit. The letters also told the financial aid 
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administrators what information they would be asked to provide during the 
scheduling calls. In addition^ a project summary was included with the 
letters to each institution (see Appendi}P: B). OPE also informed ED 
regional administrators about the study/ and provided a list of sampled 
institutions by regions. 

The Westat Telephone Research Center had the facilities and staff to 
contact all the institutions and arrange the appointments. Their corps 
of interviewers has experience in telephone survey procedures arJ 
state-of-the-art long-distance telephone facilities. Westat trained the 
scheduling interviewers in a 3-hour session at which they were 
familiarized with the financial aid process and the study. All 
scheduling calls were monitored by Advanced Technology staff on a 
rotating basis to cover all the interviewers. The scheduling 
interviewers immediately brought any problems encountered to the 
attention of the monitors. 

To schedule each visit and collect other information for our data 
collectors/ we developed a script for the telephone interviewers, an 
excerpt of which is presented in Exhibit 4-1. If an institution did not 
accept a visit on the preferred date or our alternative date, the 
interviewer asked what dates would be acceptable. The script also gave 
us an opportunity to ask about the measures of size, that is, the number 
of student recipients in each of the Title IV programs the institutions 
participate in, and to give the financial aid director at each school 
advance notice of any questions for which we would ask for copies of 
documents* 
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Mb.' 



CONTACT NAME 



NMT. COM . 
MST. NAME 



CITY 



TEUPHONC NO. 



STATE 



CALLED 

TIME 

DATE 



RESCHEDULE 



YES 
NO 



TITUE iV STUDY SCHEDULE CONnfUIATON 



QoodJrnornlno«UfiK)on)(^ . ^m—, ^lromW«rt»t Inc, 

RocMkMftfyiw)d AoQuptoofwojtniooiirSn^^ 
l«filM dMcrl*;g lh« Tlla IVQuillly Conlr^ 
lh» Idmte of acAiteft wt w« to pwformifM) 

H«vt you r«c«iv«d th» tottof and h«d a chanc* to rMd It? 



YC8 
NO 

CANT RECALL 

WANT MORE INFORMATION 



AdMUKad T6chrK)lc»oy. Inc.» and Wailit ara undar con 

Oij^Conrol8bJdyforthaTltelV8tudantAidProoram&. Tbamiiorobiadlvaaof iMtludyarato: 

• Da Wrrino paymont and award arror ratat fc>r thoaa programa by lntona««<no paranli, abxiants, 
and infttiiuiona 

• Daflna ttia probabto cauaai of thata anrora 

• Davalop corractiva action proposalt to raduca paymont arror 

T)M InslHutkMa! phasa of iNt yaar'a date colaclion ia daal^ 
totonHaw lha inancM aU adnnMa^rator. and conwila data on a larnpla of Pa^ 
radpianlaandQSLcarlMcalona. VtawM tonmkinolhaaavtailatolManFabruafy lOandM^ Tha 
Intarvtawwitakaaboutanhourandahatf. W^iMllnaadtofialactaaanriplaofyourakiraclplantalromyour 
racordsandcompilalnrom»llonont)OMtludantt. Waas^totfiatthaavaragavlaltwitekaonatotaro 
dayt^butyouwwnaodtotoavaaabiaonlytortiaintarviow. GlharatpactaofdatacoNadfondonotraquIro 
y^ praaanca. 



; ' ; " WW " ' '% X I ' '% I' Wis.. 




It tfia numbar of Pal Grant radpiants for 19e&a6? 

NUMBER 

DONT KNOW 

Whatia ma attlrnatad mirnbar of undupHcatad Campua-Saaad 

NUMBER 

ooNnr KNOW 

WiMt la lha aallmatad nun(4)ar of GSL carlllcationa for 



NUMBER 
DONT KNOW 



For planfifoo purpoaa% wa fiM MiMahad a iMi^ 
racalva our IMnoa by irM-tummar. 

thMwacftacMfotofltMnatoaakyouragardnoourviaMtoyourlnat^^ 

OurlnlarvfowarlatontBtMyachadJadtobagfotiavlstttoyourfoa^ &OOA.M. la 

lhat dato and ima accaplabla to you? 



YES 
NO 

•:00 AJi UNACCEPTABLE. What ima can our viaifor^nlva to fiat 

Inafulda/awork? 



RECORD TIME 



WtMralatwStudantAldOlllcafocatad? (RECORD BUlDtK3l. STREET ADDRESS, IF APPROPfllAI 
aOOR AND ROOM NUMBSl) ^ 



EXHIBIT 4-1 TITLE IV SCH'SDULING SCRIPT (EXCERPT) 
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Westat's telephone staff immediately reported every case in which a 
school would not agree to a visit on the preferred date to our data 
collection manager^ who rescheduled other schools for alternative dates 
if necessary. To minimize the number of changes and the number of times 
individual directors were called back/ rescheduling was, whenever 
possible, limited to schools that had not yet been reached. If a school 
did have to be called back to reschedule a visit, we used a rescheduling 
script for that purpose (Exhibit 4-2). 

A small number of institutions did not want to participate in the 
study at all, and were referred by the callers to the manager of data 
collection. She contacted the financial aid administrator at each of 
these schools and discussed their reasons for refusing and their 
obligation to participate. Those institutions that still refused were 
referred to our ED project officer. All of the 10 institutions referred 
to the Department eventually agreed to be visited. 

A few schools which were difficult to reach were called directly by 
Advanced Technology staff after the scheduling period had formally 
ended. After all schools had agreed to visit dates, we compiled a 
schedule for each data collector listing the dates and places of the 
visits. The data collectors were also provided with copies of the 
completed scheduling scripts for each of the institutions assigned to 
them. 

Several institutions had specific reasons which prevented our 
scheduling a visit on or even near the date we preferred. For some, our 
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ReSCHKDUUNG SOUPT 



H*iio, thU U . from W^sut* Inc. (l/Ont o! our othtr editors) 
pr«vioutly made an a^pointmant for our dau cotlactor to Intorvltw (you/tha tractor) 
and rtvlaw soma of your financial aid racorda as pan of tha Tltla IV Qi*aUty Control 
Study. That appolntmtnt wa« for (DATE OP ORIGINAL CONFlRMtD VOID. 

Wa hava had a schaduUng problam with anothar of tha sam^la InttitutJont in your araa. 
So that wa don*t hava to maka a iptcial trip for Jutf that KhooU wa would iika to 

chanfa tha data of our visit to your of f ica to (NfiW PREFERRCO DATE). 

CouM you accommodata us on that data? 

YES Thank you vary much. Our visitor wiii ba thtra on (NEW PREFERRED 
DATEX (END INTERVIEW) 

NO Ara thara any othar datas batwcan ^nd ____ on wlOch 

wa couid raschadula our visit? 

YES What ara tha othar datas? (RECORD DATES) 



I wiil lat our schaduUng luparvitor know about thasa altamativas 
and ha wiil gat back to you. Wa appraclata your patianca as wa try 
to work out tha bast schadula for all tha schools in tha sampla. 
(END INTERVIEW) 

NO 0.1C., wa wiil ba thara on tha data wa originally agraad to. That is 

(DATE OP ORIGINAL CONFIRMED VISIT). Thank you for your 
considaration. (END INTERVIEW) 

COMMENTS 



EXHIBIT 4-2 TITLE IV STUDY RESCHEDULING SCRIPT 
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preferred date coincided with an especially intense activity such as 
award packaging or registration/ or with spring vacation. Such 
scheduling conflicts affected other institutions in the same region, 
which required not only significant rescheduling^ but also the extension 
of the field period from 6 to 7 weeks in some regions. Such scheduling 
conflicts also required us to make extensive use of our alternate data 
collectors, since several schools had only one date open. All of our 
alternates were scheduled for institutional visits before the actual 
field period began. 

After we had agreed on a date with each institution, we sent each one 
a letter as a reminder and written confirmation. These letters also 
included a list of information which the data collector would need for 
the visit, to enable the aid administrators to gather these materials 
ahead of time. 



We required the data collectors to call each of the scheduled 
institutions in their regions to confirm the visit about a week in 
advance. As in previous data collections/ the confirmation calls 
provided an opportunity for the data collector to introduce him- or 
herself to the financial aid administrator (FAA) and to begin to 
establish the rapport that was necessary for a successful visit. Most 
data collectors reported that their calls were welcomed by the FAA*s, who 
were reassured by the data collector's clarif icz^tion of the nature of the 
site visit and review of documents and materials needed to conduct the 
visit. The calls also allowed the data collectors to learn about the 
organization and location of the office and to identify potential data 
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collection or scheduling problems, 
time for the project staff and the 
the visits. 



If problems were uncovered^ there was 
daca coiieccor to resolve thsm bsfors 



4.4 TRAINING THE FIELD STAFF 



Since all the field staff were experienced financial aid 
administrators, no special training was needed on the Pell Grant, 
Campus -Based, and Guaranteed Student Loan programs. Therefore, it was 
possible to devote the entire training session to interviewing and 
fsampling techniques and to completing correctly a Student Data Form (SDF) 
and Student Record Abstract (SRA) for each student in the sample. The 
training agenda is shown in Exhibit 4-3. 
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4.4.1 Training Materials 



We developed two manuals specifically for the data collectors: the 
Data Collector Training Manual and Question- bv-Question 
Specif ications > 

The Data Collector Training Manual covered general procedures to be 
used in the field: 

• Description of Advanced Technology and the Title IV Quality 
Control Project 

• Overview of survey research and interviewer tasks for the 
study 

• Confidentiality procedures and rights of respondents 

• Preparation for site visits 

• Conducting the interview 

• Conducting the verification of 1984-85 Pell Grant recipient 
enrollment 

• Sampling procedures 

• Procedures for completing the Student Record Abstract 

• Exit interviews 

• Field editing and reporting 

• Travel and accounting procedures 

The Question-bv-Question Specifications contained a separate section 
for each of the three principal data collection documents to be used in 
the field: the Student Data Form (SDF); the Institutional Questionnaire 
(IQ) for the interview with the financial aid director; and the Student 
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Record Abstract (SRA). Within each section, each page of the form was 
reproduced with instructions^ definitions, etc., for the items on that 
page printed on the f"..cing page. 

Because the Title IV regulations are long and complex, we could not 
anticipate in training every question which could arise in the field. 
Therefore, we provided the following reference documents to the data 
collectors: 

« Current Title IV Regulations 

• Feaeral Student Financial Aid Handbook 

• Manuals from the Student Financial Administrators Training 
Program (SFATP): 

Index of Regulations 

Aid AdministratCjr 's Guide to IRS Forms and Schedules 

• 1985-86 Pell Grant Validation Handbook. 

4.4.2 Training for the Institutional Interview 

Since our data collectors had considerable experience in reviewing 
Student Jtid Reports (SAR*s), income tax forms, and other documents in 
student aid files, but little or no experience in conducting structured 
interviews, we devoted a greater proportion of the training to the 
interview than the amount of time required for it in the field would 
suggest. 

During the first day of training, we introduced the data collectors 
to the Title IV Quality Control Project in general, focusing on its 
objectives and study methodology. We gave each data collector the 
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opportunity to practice the institutional interview in its entirety, with 
project training staff monitoring and answering questions- Our trainer/ 
monitors also demonstrated and explained some specific techniques and 
strategies which they had used successfully in their interviews in Stage 
One. 



4 •4. 3 Sampling Training 



Sampling of students was an integral gact of each site visit and/ 
therefore, an important component of training- This was also a 
challenging part of the training since few data collectors had experience 
with drawing randcm probability samples for research purposes- Westat 
staff assisted Advanced Technology project staff with this portion of the 
training- The sampling training covered three topics: general sampling 
principles; sampling lists to be provided by the institution; and use of 
the sampling worksheets - 

;^ Wastat statistician who was involved in the previous Pell Grant 

quality control studies introduced data collectors to the concepts of 

randomness and potential sources of bias. A member of the project staff 

explained the various types of lists which would be encountered and how 

to draw samples of students from them during the site visits- Data 

collectors receive! instruction and practice in the use of "clean" and 

••contaminated" sampling lists. Clean lists are developed by the 

institution and contain the names of all students in the sampling frame 

(Pell or Campus-Based recipients or GSL certifications) without 

duplication or extraneous names. Contaminated lists contain extraneous 

names, duplications, or both- The trainers introduced various techniques 
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for cleaning contaminated lists prior to sampling, if practicable, and 
offered techniques for sampling from contaminated lists if cleaning was 
impracticable. 

The data collectors were also trained to use the sampling worksheets 
to select students from sampling lists in the 'field. The draining 
session included practice sessions in which data collectors drew samples 
from various types of lists using sampling worksheets. Project staff 
reviewed the completed exercises and discussed them with the group. 
Finally/ the staff instructed the data collectors about conditions under 
which the project office should be contacted concerning sampling issues. 

4.4.4 Student Record Abstract Training 

Although our field staff had extensive experience with the student 

aid forms and validation documents from which they were to collect the 

data on individual students, they lacked sufficient experience in filling 

out and edit' g study forms ao that everyone would collect the same 

information a format that could be coded easily at the project 

office. Therefore, we reviewed the SRA in detail/ defining exactly what 

information we wanted/ how it would be used/ and what documentation was 

desired/ and discussing possible problems and answering questions. The 

most important part of the training was a series of exercises in which 

the data collectors were given student aid files for actual students 

(identity masked) from which to practice filling out SRA's. We then 

reviewed the daiia and discussed the correct way to record the data on the 

forms. The project staff also conducted a special training session on 

some of the unusual situations which might be encountered in the field/ 
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as well as a special session on proprietary institutions, since most data 
collectors came from traditional/ credit-hour institutions. 

4.4.5 Field Practice 

On the last day of training, we sent the data collectors in groups of 
three or four to seven local institutions which were not included in the 
study. Each group was accompanied by a member of the project staff or by 
a trainer/monitor. The purpose of these visits was to give the data 
collectors an opportunity to practice the interview and complete SRA's 
under conditions which more closely approximated field conditions. The 
institutions that participated in field practice represented a variety of 
types and control and sizes. 

We had arranged beforehand to interview the financial aid director at 
each practice site. Each of the data collectors did part of the 
interview. We had also asked the financial aid director to allow the 
data collectors to draw a sample of cases from their files. These were 
not a random sample, but did provide practice with real files and. at 
some siteS/ computerized record systems. 

In the evening after the field practice, we discussed and evaluated 
the day's experience. The data collectors shared the lessons they had 
learned and the project staff answered questions. 
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4«5 FIELD SUPERVISIOH 



Supervising the data collectors posed special problems because they 
were dispersed throughout the country and were moving to a different city 
at least weekly and often every few days. Monitoring each data collector 
in the field for only 1 day* each absorbed the monitors for the first 2 
weeJcs of data collection. During the remainder of the field period, most 
of the supervision had to be by mail or telephone. 

4.5.1 Monitoring the Site Visits 

The monitors, themselves experienced data collectors, visited the 
data collectors during the first 2 weeks of the field period to observe 
how they interviewed financial aid directors, selected the student 
sample, and completed SRA's. All monitoring was conducted on the first 
day of each site visit to accommodate all three components of the visit. 
Two site visits were also monitored by our ED project officer.- During 
these monitoring visits, the monitors answered questions which had not 
arisen during training, corrected some minor errors in completion of the 
SRA's, and made other suggestions to the data collectors. They found no 
systematic errors ii> the data collectors' performance of their duties. 
All monitors submitted both oral and written reports of all monitoring 
visits to the manager of data collection. 

4.5.2 Telephone Validation 

As a continuing check on the performance of the data collectors and 

to introduce a different perspective / we conducted validation by 

4-19 



115 



telephone throughout the field period. We called each of the financial 
aid directors who had been visited during the first week, except those 
where a staff monitor accompanied the data collector, and administered a 
brief questionnaire which included an open-ended question about the data 
collector's performance. For each week thereafter, we called about half 
of the financial aid directors who had been visited. In total, staff 
completed 212 validation calls, or 71 percent of all institutions. 
Through validation calls and monitoring, quality control checks were 
completed at 79 percent of the institutions visited. 

Only two problems were raised in the validation calls; neither was 
confirmed by any of the calls to the other sites visited by those two 
data collectors, which were uniformly positive. In general, the 
responses to the final, open-ended question praised the professionalism, 
knowledgeabilityf and cooperativeness of the data collectors. In 
addition, senior project staff made validation calls to those 
institutions where we had encountered any difficulty in either 
scheduling, san^pling, or file review to ensure resolution of those 
problems. 

4.5.3 Telephone Supervision 

Regular and structured communication with the data collectors 

provided the most comprehensive means of field supervision. Each data 

collector had a scheduled time at which to call the field supervisor each 

week. (Exceptions were made only in the case of an interviecr which 

interfered with the call because of a time zone differential or if the 

data collector was scheduled to be traveling.) The scheduled call was an 
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opportunity for the field supervisor to review the previous site visits 
and discuss future visits with each data collector, including prospective 
problems uncovered by the confirmation calls. The data collector could 
also raise any problems which had not required immediate consultation 
with the project staff. The field supervisor used a form that served as 
an agenda of itemc to review with the data collector and provided a place 
to record responses and problems. The weekly call also gave the field 
supervisor a chance to discuss other problems with the data collector, 
especially those which had surfaced during editing by project staff, but 
were not serious enough to demand an immediate call, such as missing SRA 
items which required a call from the data collector to the institution. 
The field supervisor also informed the data collectors about errors in 
the completion of SRA's that did not require calling the institutions for 
further data, such as inadequate field editing. The project staff used 
forms for this purpose v>.ich were placed in each data collector's file. 

4.5.4 Monitoring Field Sampling 

We provided the data collectors with three unique sampling worksheets 
for each institution. These worksheets contained line numbers, based on 
our estimates of the measures of size, that they would use to draw the 
sample from the Pell e^jd Campus-Based recipients and GSL certification 
lists at the institution. 

The data v-.ollectors carried out the Pell and Campus-Based sampling 
very well, since most institutions had recipient lists available and the 
measures of size used by the project office to generate the lists were 
relatively accurate. Where measures of size were not accural 8, the 



sampling worksheets included instructions for dealing with several 
specific problems. In those cases where the data collector coulc not 
sample from the existing list, project staff were able to resolve the 
problem. Even though we had revised our scheduling procedures, as 
discussed in Section 4.3.2, to obtain better measures of size for all 
programs than in past data collections, many institutions had still given 
us only rough estimates. Consequently, data collectors frequently 
required technical assistance from the field supervisor. This involved 
generating new sampling line numbers using the measure of size obtained 
by the data collector. This then required later revisions to the weights 
assigned to students from these institutions. Nevertheless, all site 
visits proceeded smoothly with the assistance of the project office and 
no site visits were extended because of sampling problems. 

4.5.5 Resolution of Problems in the Field 

Calls from the da^.a collectors to the field supervisor were more 
frequent than regularly scheduled calls, especially during the early 
weeks of the study. These calls dealt with unique or unusual situations 
at particular institutions that had not been covered during training or 
with questions concerning sampling and the SRA. The answer to a 
particular question about the SRA often depended on the purpose of the 
item or how the data would be used in analysis, so the manager of data 
collection often checked with other project analysts. 

The sxxbjects of all data collector inquiries and their responses were 

documented in call logs. Most frequently, the calls concerned sampling 

students at individual institutions. These problems included 
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unavailability of sampling lists and inability to draw a sample within 
the minimum and maximum numbers provided on the sampling worksheets. In 
the case of sampling list problems, various me-hods taught in the 
sampling training session were employed, primarily counting folders or 
data cards. Problems related to drawing the sample were minimized by the 
field supervisor's providing new line numbers over the telephone. These 
new sampling numbers were generated by pre-established procedures and 
recorded at the time of the phone call. 

The other major reason for calls from the field to the project 
office/ or vice versa, concerned tiie data collectors* travel 
arrangements. All field staff were required to inform the project office 
of any changes they wished to make in their itineraries , and not to make 
those changes without approval from the field supervisor. In virtually 
every case, these changes were not only for the convenience of the data 
collector, but also saved money. 

Problem resolution was also facilitated by the use of a telephone 
answering machine in the project office, for problems that arose outside 
of normal business hours. Evening messages were handled promptly the 
next morning; project staff monitored messages during weekends as well. 
These calls generally concerned unforeseen circximstances related to 
travel, such as weather ^ or questions •J^out field editing of the data. 

The availability of four alternate data collectors - people who for 
various reasons could not commit themfiielves to seven uninterrupted weeks 
of training and travel - was an invaluable resource in maintaining the 
data collection schedule* 
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4.5*6 Schedule Changes 

A major rescheduling occurred during training week, due to the loss 
of a data collector because of a family emergency. His entire assigned 
region was rescheduled for the end of the field period, which resulted in 
an 8-week data collection, rather than 6 weeks as originally planned. 
Three data collectors volunteered to work the extra time necessary to 
visit the schools in th^s region. All of the re£.cheduling and 
reassignments were completed on the same day that the original data ^ 
collector withdrew from the study. 

During the field period, several institutions asked to change the 
date of their site visit. In every case we were able to accommodate the 
request, even though in several cases chis also required rescheduling 
other institutions and reassigning data collectors. Two institutions 
were dropped from the study during the field period. Since both were 
only single-day visits, there was only a minimal effect on the data 
collectors * schedules . 

Resistance on the part of several institutions to collecting certain 
types of data or any data caused greater problems. One institution 
initially would not allow us to collect identifying information on 
sampled students maintaining that university policy required permission 
from the students prior to their inclusion in the study. After several 
discussions between the ED project officer and the FAA in which the FAA 
was informed of the exclusion of financial aid records from protection 
under the Privacy Act, the institution permitted the data collector to 
proceed. 
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Several institutions that had initially declined to be visited were 
not in the original master schedule, since it took some time to resolve 
these cases. The master schedule did. however, contain open dates to 
accommodate these institutions once their situations had been resolved, 
either through project staff or the ED project officer. All of them were 
eventually persuaded to participate in the study, and as each of these 
cases was resolved, the schedule was adjusted and a data collector was 
assigned. This resulted in some rescheduling of institutions throughout 
the field period. 

One institution which had originally refused later complied with the 
visit. This was one of the two instances where the institutional 
interview - which was voluntary - was not conducted. The lack of 
cooperation and assistance experienced by the data collector caused this 
visit to be extended in order to complete the sample. 

4.5.7 Field Expense Reporting 

Advanced Technology instituted a set of cost-reimbursement policies 

and explained them to the data collectors during training (Exhibit 4-4). 

Data collectors were required to submit comprehensive daily expense 

statements with receipts and to maintain personal daily expense logs. 

When these daily reports were late or slow in arriving, we called the 

data collectors. In a few instances, the field supervisor asked about 

unusual expenses during weekly calls; she also notified data collectors 

about non-reimbursable expenses at that time. Non-reimbursable expenses 

consisted mostly of meals and personal telephone calls which had gone 

over the weekly limit or other personal expenses. 
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COST REIMBURSEMENT POLICIES 



4-26 122 



We required the data collectors to notify us of travel changes in 
advance. Once they reached the appointed cities, some data collectors 
found hotels that were both closer to their schools and cheaper than the 
ones that we had been able to reserve at a distance. 

4.5.8 Field Staff Reports 

At the end of data collection, we asked each data collector to submit 
a written field report covering a list of topics we supplied. Some of 
the reports were quite detailed, and the data collectors were thoughtful 
and constructive in their comments. 

The field reports indicated that many of the procedural revisions we 
had made since Stage One had resulted in improvements in the data 
collection. We had increased our efforts to give the institutions more 
information to enable them to be better prepared for the visits, and we 
were more assertive about getting better information from them regarding 
measures of size to improve student sampling. These two changes meant 
that the visits were conducted much more efficiently than in the past. 
Changes in the format of the sampling worksheets and in the instructions 
on using them resulted i fewer calls to the project office for sampling 
assistance. Changes in the format of the Question-by-Question 
Specifications (Q by Q*s) ensured their extensive use in the field in 
Stage Two. 
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Some aspo-ts of the data collection did not change, however. The 
instruments, the Institutional Questionnaire (IQ), the Student Data Form 
(SDF), and the Student Record Abstract (SRA), took about the same amount 
of time to administer as before - about 1 1/2 hours for the interview, 
about 15 minutes for the Student Data Form, and almost an hour for each 
SRA. These constants indicate that many of the factors that affect the 
efficiency of data collection are dependent upon the institutions 
themselves. Also, despite individual variations, our data collectors 
encountered a consistent range of: cooperation from FAA's, ease or 
difficulty of sampling, and level and appropriateness of file 
documentation. 

Most directors of financial aid were well prepared for the visit and 
interested in the study, although many expressed some anxiety about being 
visited. The data collectors were sensitive to this and tried to do 
their job as quickly and efficiently as possible. The confirmation calls 
helped the FAA's to understand the nature and purpose of the visits, as 
well as the data collectors to understand context and characteristics of 
the financial aid office. 

Most of the data collectors' reports expressed satisfaction with 
training, despite the intensity of four very long days. Field practice 
was, again, a very important component of training. The field staff 
appreciated the efforts of the staff in turning them - at least 
temporarily - from practitioners into researchers. 

They also expressed satisfaction with the level of support and 
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responsiveness of the project staff while they were in the field. The 
data collectors' overall impressions of their field experiences generally 
focused on their receptions by the institutions, as noted above, as well 
as their own observations about the efforts and quality of the financial 
aid offices' operations. Most had the opportunity to view a wide variety 
of situations, gain an understanding of program intent and procedures, 
and develop a sense of commitment to program quality and quality control. 

4.6 QUALITY CONTROL OF THE FIELD DATA 

In order to ensure the quality of the institutional data, we 
implemented sevsral procedures to review the incoming forms and correct 
any problems which might have occurred. These review procedures began 
during the field period, and continued until the data were considered 
"clean" and ready for analysis. Exhibit 4-5 shows -the various quality 
control procedures we used. 

4.6.1 Field Editing 

The data collectors were instructed during training to review all 
materials after each site visit and to edit for completeness, accuracy, 
and legibility before mailing. This provided them with the opportunity 
to correct any coding errors they may have made, such as not following 
skip patterns correctly, zero-filling, or justifying margins. Field 
editing also allowed the data collectors to make additional marginal 
notes to explain institutional procedures and individual circumstances, 
and to clarify explanations of open-ended or "other" responses. 
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QUALITY CONTROL OF FIELD DATA 



4.6.2 Project Staff Review 



As soon as the data collection materials began arriving from the 
field/ project staff r^vieweu the work of all the data collectors. These 
reviews/ together with the monitors* reports / formed the basis of the 
early supervision callS/ wherein the manager of da^ia collection discussed 
any items that needed correcting/ and gave the data collectors general 
feedback about the quality of their work. 

The data collectors were told during training that they were our 
primary contact with the institutions during the field period. 
Consequently, any questions that arose about data from the SRA's were 
referred to them for resolution. In most caseS/ the data collectors were 
able to answer these questions immediately for the field supervisor; if 
not/ it was their responsibiliny to call the institutions. Calls to 
institutions for data retrievp.i frequently concerned information on the 
St^'lent Data Form, which was often very difficult to obtain. Financial 
aid offices do not consistently maintain locator information for their 
students or the students* parents; in addition/ the accuracy of that 
information was difficult to verify. 

The coding supervisor also reviewed every SRA after the initial round 
of coding. She referred any problem cases for discussion with the data 
collectors. Forms from approximately 95 percent of the institutions 
visited underwent this review before the end of the field period. This 
procedure proved to be very effective in resolving many questions before 
the data entered the editing phase. 



4.6.3 Data Editing and Data Retrieval 



The coding and editing process, which will be discussed in greater 
detail in Chapter 6/ was designed to include several quality control 
checks on the data. Each coder was assigned the work of specific data 
collectors, so that any consistent problems with following instructions 
would be more noticeable. The coders also batched the instruments by 
institution, coding the Institutional Questionnaire (IQ) at the same time 
as they coded the Student Record Abstracts (SRA). 

The coding staff were also able to use many of the documents obtained 
during the site visits to understand and resolve questions about the data 
that arose during the coding and editing process, but after the field 
period. 

Many questioiJ.s required a wider knowledge of student financial aid 
and the Title IV programs, and were referred to project analysts for 
resolution. A small percentage of the cases could not be resolved by 
project staff, however, and in these cases it was necessary to call the 
institutions for data retrieval. Nfo specific item, or group of items, 
required data retrieval more frequently than any other. We were able to 
minimize, through in-house editing and staff reviews, not only the total 
number of cases which required institutional data retrieval, but also the 
burden on the institutions which did have to be called. If it was 
discovered that a call was necessary for one case, then all the cases 
from that school were reviewed before the call was made, to prevent a 
series of calls to one school. 



Since data retrieval by telephone was a potentially sensitive issue, 
the calls were made only by designated staff members. Most were made by 
the same person who had done the validation calls, and was therefore 
somewhat familiar to the financial aid administrators, A number of calls 
were also made by the manager of data collection, particularly if there 
was reason to expect some resistance to data retrieval; she was able to 
respond to concerns, answer questions, and obtain the necessary 
iurormation. Virtually all the FAA's who were called were fully 
cooperative with our requests and data retrieval was completed smoothly. 
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STUDENT, PARENT, AND SECONDARY DATA COLL£CTi(^ 



The methodology used in the Title IV Quality Control Project depended 
upon the analysis of data acquired from various sources related to each 
sampled student's financial aid award. Data from each of the sources was 
used to verify or refiite the amount of aid that each sampled student 
received. In addition to the individual-level data abstracted from the 
financial aid files at each of the sampled institutions^ information was 
also obtained from the sampled students themselves^ their parents^ the 
Internal Revenue Service (IRS)^ financial institutions, and tax 
assessors. The data from students, parents, and the secondary sources 
was collected by Westat, under a subcontract to Advanced Technology. 

This chapter discusses the student and parent survey, and the 
secondary data collection, including the organization of the field staff; 
supervisor and interviewer training; student and parent interviews; and 
the Automated Survey Control System (ASCS). 

5.1 ORGANIZATIC»l OF THE FIELD STAFF 

Conducting a national-scale survey of the scope of the student and 
parent data collection required close supervision by experienced field 
supervisory staff. Five supervisory regions were set up, with each 
region containing roughly the same number of interviewers. Exhibit 5-1 
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shows the five regions. The field network for the student and parent 
survey was constructed so that the five supervisors assigned to the 
project could effectively manage the 100 interviewers hired and trained 
to complete the interviews. The field supervisors were regionally based, 
one supervisor and an assistant in each region. The number of sampled 
stuQonts in each region varied from about 1,000 to almost 1,500. The 
reason for the variation in the number of sampled cases by region is that 
in some regions, the cases were more dispersed than others, requiring 
more interviewer time per case than in regions where the cases were more 
clustered. 

The supervisors were in almost daily contact with the interviewers, 
reviewing the progress on cases that had been assigned and assigning new 
cases as needed. The supervisors reported to the Westat field director, 
who carefully monitored progress in all regions, and participated in the 
decisions regarding travel and reassignments . Supervisors were also in 
frequent contact with the data preparation manager on specific cases and 
for quality control. Also, frequent communication was required between 
the supervisors or their assistants and the project programmer, so that 
the Automated Survey Control System (ASCS) could function properly. 

The supervisors and their assistants recruited and hired the 
interviewers during February 1986. Of the 100 interviewers hired and 
invited to training, most (64 percent) had worked on at least one 
previous quality control survey (BEOG, Pell Grant, or Title IV Stage 
One). They were recruited and selected from interviewer records, which 



were sorted by geographic are;*. Although we would not know the actual 
location of the sampleri students until much later, we did know the 
institutions which were in the sample/ and established the field plan 
based on our estimates of the likely number of students who would be 
sampled at each institution* 

5.2 SUPERVISOR AND INTERVIEWER TRAINING 

Training for field supervisors and interviewers was conducted by 
Westat project staff . Two training teams / of three trainers each, were 
used to conduct two concurrent interviewer training sessions* The lead 
trainer on one team was the same person who had designed the session, and 
the other was the Westat field director* The field supervisors and the 
assistant field supervisors were also experienced trainers, and served as 
"community leaders" in the training sessions. 

Home study materials had been sent to the. interviewers' homes prior 
to the training session* They were required to review these materials 
and to complete an exercir)e on the materials to hand in at training* All 
of the interviewers were given an interviewer's manual and information 
packet when they arrived at the training site. The two interviewer 
training sessions were conducted in Tampa, Florida, and San AntoniO/ 
Texas in late March 1986* Less experienced interviewers, and a few 
experienced interviewers who felt the need for review, attended an extra 
session on general interviewing *-echniques* 

The supervisors and assistants were also trained on the operation of 
the new PC -based ASCS equipment and. procedures* The two project 
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programmers who had designed the system were sent to the training sites 
to work with the supervisors and assistants on the new ASCS. This 
training was done the day before interviewer training began^ and in the 
evenings until the field staff was familiar with the operation of the 
ASCS and the equipment. 

The first day of training for all interviewers included an overview 
of the study and of the Title IV programs^ and an overview of all of the 
procedures and materials that would be used for the survey (see Exhibit 
5-2). Also included was a role play session on the student 
questionnaire^ and a discussion of the role play session. The second day 
covered the >^uestion-by-question specifications for the student and 
parent questionnaires, and handling special problems like refusals and 
answering respondents* questions. Lecture sessions were done interactively , 
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interactively / with an overhead projector to display the sections of the 
questionnaire which were being discussed. On the third and last day of 
training^ the question-by-question specifications for the parent 
questionnaire were completed^ and administrative procedures were 
presented. A final examination was given at the conclusion of the 
session. All of the interviewer-trainees successfully completed training 
except for two, one at each training site. Both were dismissed* 

There were three areas of the country where , after the student sample 
began to be received/ a need to train additional interviewers was 
identified. The first additional training session was held in Boston^ to 
train three interviewers for the northeast. Another interviewer was also 
trained to work in the southern region, and another to work in central 
Pennsylvania. 

■^ased on both their performance in training and the quality of the 
completed questionnaires that were receiveij from the field, the training 
sessions prepared the interviewers well for their work. Even those 
interviewers who had never worked for Westat before did not have 
significant quality problems. 

5.3 STDDENT AND PARENT INTERVIEWS 

The sample selection process yielded a nationally representative 
sample of students receiving Pell Grants, Campus-Based awards of various 
types, or Guaranteed Student Loans. The basic locator information for 



these students and^ in most cases for their parents^ was obtained by 
Advanced Technology data collectors who visited the institutions during 
February. March, and April 1986. Personal interviews with these sampled 
students anrl their parents were needed to confirm demographic and 
financial data about the students and their families and to obtain 
documented confirmation of information that had been reported on their 
aid applications. 



5. 3.1 Receipt of Samples from Advanced Technology 

As each student was sampled at the selected institutions, the 
Advanced Technology data collector completed a Student Data Form (SDF) 
which contained basic contact information on the sampled student and the 
student's parent (if such information was contained in the files at the 
institution, see Exhibit 5-3). These completed SDF's were sent back to 
Advanced Technology, where they were edited, copied, and forwarded to 
Westat to be processed and sent into the field as student and parent 
assignments. The processing at Westat involved keying the SDF's and 
entering them into the receipt control file. Computer-generated mailing 
labels, questionnaire labels, and "mini-labels'* containing only the ID 
number were then produced. Assignment case folders were assembled, 
consisting of a call record form for each case, with both a student and a 
parent label on the cover, and extra labels and mini-labels stapled to 
the inside. An advance mailing package, to be sent to each sampled 
student and parent, was also assembled at this time. 
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An addition to the identifying information that was included on the 
label was an identifying for Pell recipients. The "P'* was added so 

that the interviewer would know which of the cases assigned to him or her 
were Pell Grant recipients/ for whom some of the questions in the 
questionnaire were mandatory. 

5.3 .2 Case Ass igisments 

Receipt of SDF's at Westat for processing and case assignment 
depended upon Advanced Technology's receipt of materials from the 
institutional data collectors. The student and parent data collectior^ 
was scheduled after the institutional data collection, and ran from early 
March through early June 1986. The highest voliime of interviews per week 
occurred from late April through late May, as respondents were contacted 
and scheduled to yield a backlog of assigned cases. All cases had been 
assigned by late April • 

Assigiiments to interviewers were made by the field supervisors . 
Usually, the supervisors tried to keep interviewers supplied with 1 
week's workload. In the early part of the field period, assignments were 
uneven due to delays in receiving the SDF's. Later, however, each 
supervisor tried to adjust the workload of all interviewers so that they 
had enough cases to work as many hours each week as they were scheduled. 
The allocation of assignable cases had to be done carefully, because 
interviewers worked different hours and often on different schedules. 
For example, some interviewers worked only 20 hours each week, while 




others worked full time, and some interviewers worked in the evening and 
on weekends/ while others had no such preference. 

In an effort to reduce the impact of the delay in getting assignments 
out to interviewers/ the interviewers were instructed to conduct the 
student interviews first/ saving the parents until later in the field 
period. This was done so that the students could be interviewed before 
they left school for the summer. Sometimes/ however/ an interviewer 
arrived at the home of a parent for an interview and found that the 
student was also there. The interviewers were instructed to conduct the 
student interview then as well, even if the actual assignment folder was 
with another interviewer or even in another region. This procedure often 
required adjusting the recordkeeping for individual cases. The 
supervisors carefully monitored these situations/ coordinating with both 
the Westat field c'irector and the other regional supervisors when other 
regions were involved. 

The parents of independent students were initially contacted by the 
field supervisors/ and interviewed over the telephone. If during the 
telephone interview it appeared that the parent was actually the parent 
of a dependent student/ the case was reassigned to an interviewer for an 
in-person interview. 

5. 3,3 Advance Hailings and Interviews 

As the cases were prepared for mailing to each of the regional 
supervisors/ advance mailing packages were also assembled and sent to 
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each sampled student and parent. The first mailing was sent to the 
"current'* address of the student, and to the parent at the student's 
reported "permanent" address. For most students # the current address was 
their school address^ while their permanent address was their parents* 
address. The advance mailing included a letter explaining the study and 
soliciting the respondent's cooperation, a list of frequently asked 
questions, and a checklist showing the respondent the items that would be 
most helpful to have during the interview, such as copies of tax returns/ 
rent receipts, medical and dental bill receipts, and so forth. 

Cases were assigned to interviewers at the same time that these 
materials were being mailed. After an interviewer was given case 
assignments, he or she contacted the respondent by telephone to arrange 
an appointment, and answer any questions that the respondent migh-t.have. 

Generally, the interviewing went very well. A 10 percent sample of 
respondents were contacted by telephone as a validation procedure; the ED 
project officer also monitored two interviews. No problems were 
identified as result of these activities. As always, there were a few 
troublesome questions that were difficult for the respondent to 
understand. The questions that caused the most difficulty were the 
questions on taxable and nontaxable income, and the questions on 
household size and number in school; this was consistent with our 
experience in previous QC studies. Apart from these items, the problems 
with the questionnaires were minor. 




5.3.4 Nonresponse Follow-up 



The students who were selected for interviewing in the Stage Two 
survey were recipients of all five Title IV programs. For the Pell Grant 
recipients/ participation in the survey was not strictly voluntary. 
While Pell Grant recipients were not required to answer all of the 
questions in the student questionnaire, they were required to provide the 
sane information as required by the verification regulations. For 
sampled students who did not have Pell Grants, participation in the 
survey was voluntary. Moreover, at the direction of the Office of 
Management and Budget (0MB), a strong statement to that effect was added 
to both the student and parent questionnaires, which was read to 
respondents at the begirjaing of each interview. Even with these cautions 
included in the questionnaires, the refusal rate was about 7 percent, 
with total nonresponse frc.i all causes at about 10 percent. This 
compares well with the results of previous QC studies, where the response 
rates were consistently around 90 percent in the earlier Pell Grant 
studies, and slightly below 80 percent for Stage One of the Title IV 
study. 

The two main efforts employed in the field to minimize nonresponse 
were first, a vigorous refusal conversion program, and second, thorough 
tracking and tracing for hard to locate respondents. Both of these 
efforts were undertaken by the field supervisors, with the support of the 
Westat project office. 

ScuCaats and parents who refused to be interviewed on the first 
contact from the interviewer were sent a letter, either from the Westat 
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project office or from the regional supervisor, explaining the need for 
the data, and asking tham to reconsider. Also included in the letter was 
a toll-free telephone number that the respondents could use if they had 
questions. After about 1 week, the field supervisor for the region where 
the "refuser" lived followed up the letter with a telephone call, again 
asking for the respondent's cooperation. If the supervisor was 
successful in converting the refusal, the case was immediately assigned 
to another interviewer to conduct the interview. If the conversion 
attempt was not successful, the case was closed out as a final refusal. 

Initial tracing was done by the interviewer to whom the case was 
assigned. Tracing sampled respondents was also a part of the field 
supervisor's responsibility. Tracing began with data gathered by 
Advanced Technology data collectors during institutional visits. If 
parents could not be located through this information, more data were 
sought from former employers, etc. In most cases, the students required 
more tracing efforts than the parents, who tended to have more stable 
addresses. If the interviewer could not locate the respondent, the 
supervisor called the student ' s institution to ask for additional 
information that might lead to the respondent. These efforts were 
usually successful. While it was often difficult to schedule 
appointments with students because they were seldom at home, it was 
usually not difficult to find them, except at the end of the data 
collection period when they were more likely to have left the area where 
the institution was located. 
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5.4 AUTOMATED SURVEY COHTROL SYSTEM 



Progress during the field data collection was monitored using the 
Automated Survey Control System (ASCS), a computer-based information 
system designed by Westat for use in the Title IV survey. The ASCS used 
personal computers (PC's) located in each of the field supervisors' 
offices to track the status of each student and parent case, and to 
compile suimnary reports which would allow the project management to 
monitor the entire data collection activity. An example of the ASCS 
reporting format is shown in Exhibit 5-4. There were seven basic 
procedures, as shown in Exhibit 5-5, programmed into the PC's, guiding 
the supervisors through each task: 

♦ Assign cases to interviewers 

♦ Enter and update disposition codes 

♦ Enter interviewer hours 

♦ Record interviewer expenses 

♦ Reassign cases to another interviewer 

♦ Transfer cases out of a region if new address information is 



♦ Produce weekly productivity and expense reports for the 
project office. 

In addition, the ASCS could be used to produce special tabulations 
using any of the variables in the system. 

5*4.1 Production of Weekly Progress Reports 

ASCS was planned to operate within a procedural framework that 
allowed for weekly reporting and reduced to a minimum the time lag 
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between actual work completed and reported work completed. The five 
regional supervisors^ using the procedures outlined in their manuals , 
called all of their interviewers on Tuesdays ^ and updated their 
production and expense information as a result of those calls. On 
Tuesday nights, the supervisors left their PC's on, so that files from 
the regions could be transmitted to the computer in the Westat project 
office, which would then generate the progress reports used in th3 
project office, update the files, and transmit them back to the regions 



on Wednesday night. These reports were available to Westat project 
management on Wednesday morning/ and reflected work that was completed in 
the field as of Monday evening. 

Following the completion of the student and parent data collection, 
the supervisors shipped their computers to Westat (and separately sent 
their floppy disc files as a backup). The ASCS files were then matched 
with the receipt control so that all of the cases were accounted for in 
both systems. 

5.4.2 Use of Weekly Progress Reports 

Westat produced five weekly reports using the ASCS system. Reports 
1/ 4, and 5 were for the use of the. field supervisors: 

♦ Report 1: Weekly interviewer report 

♦ Report 4: Interviewer disposition report 

♦ Report 5: Region disposition totals report. 

The interviewer report provided a list of all current cases assigned 
to each interviewer in the region. The list indicated how long the cases 
had been active so that the supervisor could take steps to brl.-^g them to 
completion if they had been active too long. The report also included 
the disposition codes for all cases/ and hours and expenses for the 
supervisor's review. 

The interviewer disposition report compared interviewers , case 
dispositions/ response rates / and total hours and expenses. This 
information was used to evaluate the different interviewers* productivity 
Er|c 5-17 

14S 



and expense records and identify any situations outside acceptable 
ranges. 

Report 5 gave the region disposition tota's and let the supervisor 
know how each region was doing on an aggregate basis. Westat project 
staff reviewed these data with the field supervisor by telephone each 
Wednesday. The supervisor was advised of progress in other regions and 
what possible actions might be taken to improve the individual region's 
progress . 

The two remaining reports were for Westat project management: 

- Report 2: All-region productivity rates 
• Report 3: All-region cost "summary. 

The productivity report was actually a series of eight reports giving 
dispositions of respondents classified by student/parent, dependent/ 
independent and overall rates. These data were reviewed by the Westat 
field director and project manager and then condensed for a weekly report 
to Advanced Technology and ED. 

The weekly cost report included \ summary by region of total hours, 
total expenses, cost per completion, and other items useful as field 
survey management tools. These data were approximate, beinj updated 
later by actual time and expense reports, but provided the timely review 
of costs necessary to guard against variance from budget and 
unanticipated costs. 



5.5 SECQNDAHY DATA COLLECTION 



The methodology that was used in the quality control studies relied 
upon both data that was acquired during interviews with students and 
their parents^ and information that was acquired from secondary sources* 
The secondary data was used to verify the information that was contained 
on the students* applications for Federal student aid^ and to provide an 
additional source of documentation for information that may not have been 
readily available during the student or parent interviews. The most 
important of these secondary sources was the Internal Revenue Service 
(IRS). Students and parents were asked to sign release forms (IRS Form 
4506)/ to request the IRS to provide us with copies of their 1984 income 
tax returns. However ^ data were also collected from banks and savings 
institutions on respondents* account balances as of the date of 
application^ and from tax assessors on the value of the respondents* 
homes. The home value assessment was done for a 25 percent subsample of 
institutions in the study. For all those respondents from this 
institutional subsample who reported owning a home^ we requested tax 
assessor forms. 

5.5.1 IRS Tax Returns 

The most important component of the secondary data collection was the 
acquisition of a copy of each student's and parent's 1984 tax return 
directly from the IRS. This data source was particularly important 
because the 1984 tax return represents the most objective source for 
verifying a variety of items that were included on the student's 
financial aid application. Since the copies of the 1984 returns were 
sent to Westat directly from the IRS Service Centers^ there was no chance 
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for the respondent to alter the return to reflect what was initially 



Arrangements were made with the IRS to obtain copies of taxpayer's 
returns using the same basic procedures that were used during previous 
rounds of data collection on the quality control studies. Agreement was 
reached with the IRS central office on the procedures that would be used 
to handle the requests. The central office notified the individual IRS 
Service Centers about these procedures, since they procesoed the requests 
and forwarded the copies of the returns to Westat. 

The procedures that were established by the IRS first required that a 
release form, IRS Form 4506, be signed by the taxpayer authorizing Westat 
to receive the copies of the 1984 returns directly from the Service 
Centers. The 4506 forms that were used were modified for the Title IV 
study by filling out in advance the items specifying where the copies 
would be sent and the kind of information requested. A 4506 form, with 
instructions, was sent to each sampled student and parent in the advance 
mailing package along with the other materials explaining the study. 
Later, when the interviewers conducted the interviews with students and 
parents, they asked the respondents to again sign a 4506 that was bound 
into the questionnaire. While this procedure resulted in a great many 
duplicates, it also provided for a double chance of obtaining the signed 
form. 



r^eported on the student's application for financial aid. 




When the signed forms were received at Westat, either directly from 
the respondent using the copy of the 4506 form that had been included in 
the advance mailing, or from the questionnaire itself, the forms were 
logged in, separated according to the Service Center where the respondent 
had filed his or her tax return, and put into batches. A specially 
designed transmittal form was also filled out for each shipment. 

Early in the field period, we were notified by the IRS that a routine 
review of these procedures by other branches in the IRS had produced some 
concerns about whether study participants were coerced into signing the 
4506 forms. Specifically, the IRS was concerned that Pell Grant 
recipients, who were already required by program regulations to provide 
verification of certain application items, could construe our request as 
coercing them into releasing information in order to obtain student aid 
awards. Even for voluntary participants, there was concern that the 
respondent would feel coerced. We were therefore instructed to hold all 
of the 4506 forms that had been collected until the IRS could fully 
review the issues with their legal staff. 

After careful review and discussion with ED project staff, and 
documentation of study procedures which provided adequate information of 
participants informed consent, IRS officials complied with our request to 
process the 4506 forms. We were instructed, through the Department of 
Education, to send all of the 4506 forms that had been received to the 
IRS main office. The IRS sent the completed forms to the 10 Service 
Centers, except for those forms which the IRS determined not to be clean 
enough to process. In most of those cases, thf) form either did not have 
a date of signature, or the date was unclear. 



The 4506 forms which were rejected by the IRS included those which 
had been held beyond the IRS 60-day limit on processing requests for 
copies. These forms were returned to the respondents with a letter 
thanking them for their participation^ and asking them to resign and date 
the form. A reminder letter was also sent, along with another copy of 
the 4506 form. Of the 490 forms that were sent back to the respondents 
for clarification/ 289 were returned to Westat. 

A total of 4/278 unduplicated 4506 forms were sent to the IRS. 
Usable copies of tax returns were obtained for 2,830. The IRS performed 
a system search for 1/408 returns / and confirmed that those respondents 
did not file tax returns in 1984. The 40 remaining forms had problems 
such as an incomplete signature/ or a spouse signing the form for a 
single taxpayer who filed separateli'/ oi' an incorrect Gocial Security 
number entered on the form. Since these problems with the forms occurred 
very late in the data preparation/data processing cycle, there was 
insufficient time to recontact the respondents to correct the problems 
and re-enter the data into the file. 

5. 5. 2 Financial Institution Records 

As a part of the interview/ respondents were asked about the amounts 
they had in savings and checking accounts at the time that they completed 



their financial aid application. If the respondents said that they 
didn't know/ or if they said that they had more than $4/000 in checking 
and savings, the interviewer asked them to sign a release authorizing 
Westat to get account balances from their banks and savings and loan 
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l offices as of the date of their application. An authorization form was 

I prepared for this purpose, and bound into both the student and the parent 

\ questionnaires (see Exhibit 5-6). A separate release was obtained for 

each account. 

The releases were sent to Westat with the completed questionnaires 
and other documentation. The forms were logged in, checked for claricy 
and comple*- ^ness, and sent to the appropriate financial institution with 
a cover letter explaining the study and asking for their help. Telsphone 
follow-uo was an important part of this particular data collection, with 
the institution frequently simply giving the information to the Westat 
telephone interviewer over the phone. 

A totcil of 180 r6?^eaf;es wero sent to financial institutions. Usable 
responses were received for 135 respondents. As with the 4506 forms that 
were sent to the IRS, some of the release forms could not be honored by 
the financial institution. In these cases, the signature did not match 
the signature on the account, the account number was incorrect, or the 
financial institution had no record of the account. 

5.5.3 Tax Assessntent Records 

As estimate of home value was asked in both the student and parent 
questionnaires for those who owned homes. A subsample of 25 percent of 
sampled institutions was selected, and all of the respondents from among 
those cases who identified themselves as homeowners were included in the 



CASE ID # 

Authorization to Release Information 



U.S. OttpartiB«nt of Education 
Title IV Quality Control Study - Stage ll 



TO: 



Name of Financial Institution 



Branch of Financial Institution 



Address 



Kainefs) of Account Holder (9) Agggun^; Nmnb^g 



I hereby request that you complete and return, directly to 
Westat, Inc., the following information regarding the balances in 

my/ our accounts. 

Balangt as <?g L.J 



( Ottt AppStcation Signed ) 

A copy of this request is provided for your records and a 
return envelope, postage prepaid, is enclosed for your 
convenience. 

Sincerely, 



Signature of Account Holder Date 



IF JOINT ACCOUNT 

MUST HAVE BOTH — • Signature of Account Holder Data 
SIGNATURES 



EXHIBIT 5-6. AUTHORIZATION TO RELEASE INFORMAT'lN FROM FINANCIAL INSTITUTIONS 



homeowners subsample. The tax assessors for the jurisdictions where the 
homes were located were contacted by mail to request the latest assessed 
value • 

We mailed out requests for assessments for 232 properties. A 
considerable amount of research was required to identify the locations of 
some properties^ and their appropriate assessment jurisdictions. 
Extensive telephone follow-up was required to obtain some assessments. 
Once the tax assessor was successfully contacted, the assessment 
information was frequently supplied over the telephone. Assessments were 
obtained for 156 properties. 

The secondary data collection thus provided us with important 
additional information to confirm and complement the data obtained from 
the primary sources - students, parents, and institutions. These data 
were then coded and edited, providing a consistent format for merging the 
data from all sources into a single master file for analysis. 
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DATA PREPARATION AND PRCXIESSING 



The preparation and processing of data collected in the field by both 
Advanced Technology and Westat required a nximber of cteps and operations 
in order to produce a set of clean data tapes, ready for analysis. 
Advanced Technology assumed this responsibility for the institutional 
data (institutional interview data and Student Record Abstracts) while 
Westat was responsible for providing the clean data for the student and 
parent interviews and the secondary data (IRS forms , financial 
institution records , and tax assessor records). Advanced Technology 
performed the necessary merging of these data files to produce a complete 
master file for Stage Two. 

6.1 DATA SECURITY 

The success of the study depended on obtaining and verifying 
sensitive information about the personal and family finances of aid 
recipients and their parents. Therefore, guarding the privacy of the 
data acquired was an important element of the study. 

To ensure that the data collected were not available to anyone 
besides authorized project and ED personnel, a set of standard 
confidentiality procedures were followed. 

♦ All employees signed an assurance of confidentiality. 



J 

• ' Employees kept completely confidential the names of 
respondents, all information or opinions collected in the 
course of interviews^ and any LaSq^- ntion about 
respondents learned incidentally. 

• Unless specifically instructed otherwise, an employee or 
field worker, upon encountering a respondent or 
information pertaining to a respondent that he knew 
personally^ immediately terminated the activity and 
contacted his supervisor for instructions. 

• Survey data containing personal identifiers were kept in 
a locked container in a locked room when not being used 
each working day in routine survey activites. Reasonable 
caution was exercised in limiting access to survey data 
only to persons working on the project who had been 
instructed in the applicable confidentiality requirements 
for the project. 

• The project director was responsible for- ensuring that 
all personnel and contractors involved in handling survey 
data on the project were instructed in these procedures, 
signed the pledge, and complied with these procedures 
throughout the period of survey performance. 

• The project director ensured that survey practices 
adhered to the provision of the U.S. Privacy Act of 1974 
with regard to surveys of individuals for the Federal 
government. 

e Selected coders and editors kept the completed 
questionnaires in the check-in area. Special handling 
instructions not only demanded enforcement of the 
confidentiality of received questionnaires, but assured 
strict control of questionnaire whereabouts. 



Data collection procedures included the assignment of study 
identification numbers to all sampled institutions and students in order 
to ensure the confidentiality guaranteed by the Department of Education 
to study participants. Advanced Technology project staff assigned these 
nxunbers to the institutions as soon as the institutional sample was 
drawn. The field staff assigned identification nxxmbers to the sampled 
students at the time of sampling. Institution and student names were not 
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entered onto the automated files which were used for analysis and 
eventually delivered to the Department of Education. 

Because of the extensive merging of different files required for the 
project, one individual identifier— Social Security number—had to be 
retained on individual records. However, name and address information 
was kept only cn Westat*s receipt control files. All other files resided 
at ED'S contract data processing facility. Access to these files 
depended on having both the account number and a current password. 
Passwords were changed frequently— at least every 4 weeks— and their 
dissemination strictly limited to data processing staff. Analysts not 
directly involved in file revision or manipulation did not have the 
passwords . 

All hard-copy files (original paper forms) are stored in a locked 
room for the duration of the study, and will be destroyed according to 
approved procedures after completion of the project and upon instructions 
of the government project officer. Remaining identifying information 
will be deleted from the computer files at the same time. 

All emj-loyees of both Advanced Technology and Westat who had access 
to information about individual students or parents signed 
confidentiality pledges that they would not reveal any of the information 
they acquired or saw in their work to anyone not involved in the study. 
The training for both Advanced Technology field staff and Westat 
interviewers included a briefing on privacy requirements and research 
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ethics. Both firms assigned to the study experienced researchers who 
were well aware of privacy requirements^ most had wor^kod with comparable 
data on previous studies » The few new employees were also briefed on 
privacy requirements. The confidentiality pledges are shown in Exhibits 
6-1 euad 6-2. 

6.2 AOTOMATKD RECEIPT CCaiTROL 

We kept careful records of each individual data collection document 
received from the field. These records included the status of each 
document as it progressed through the various data preparation steps# 
including the date each step was begun and completed. 

6.2.1 Institutional Data 

For the Stage Two institutional data collection, we used an automated 
receipt control system, using LOTUS 1-2-3 database management and 
spreadsheet software. This enabled us to track the flow of the survey 
instrtaments from the time they were received from the field through the 
data editing cycles until we had complete, clean data files, as shown in 
Exhibit 6-3. 

Our receipt control clerk examined the contents of each package for 
completeness, verified that each Student Record Abstract (SRA) was 
accompanied by a Student Data Form (SDF)/ and compared the information on 
each for correct study ID number and S cial Security number. The 
Institutional Questionnaire (IQ) and SRA's were then assigned batch 
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AOVAHCCO niCNNOtOGY» INC 

UM^YU Ot CONTMC1WS ASSUiAHCC OT OONTOeNTIALITY 
Of SimViYMTA 

STAmwHTorroucY 

MvMC«4 TadiAoioiy U f kmly ommMiM to tht prlAclpIt that tht omfMifiiUI. 
ily 1 iMftirMMt 4iu m%imd tHrMg^ A4tf«fic«4 T«clii»bgy Survtyt mmt U 
^ol«cu4. THU princlfkk Mit «tMtl)«r «r not My tp«cilk guvMlM of conlM^MUI- 
Ily tMt glv«i M tliM •! initrvtew (or MllHrti|N»MK m wMkt or Ml Hwrt ar« 
tfMCiilc cimUKtiiil •bttftikMt 8»it clUnt. Vlim fiMrMtatf Hiv* ^ gIvM or 
coMracliMl McatkMw t^m4ki% confklinlUlliy lim koin Mtrotf kilo. Uity may 
Impoio oMikMwl rtfukomMUi whldi art to ko odhtrtd to tiHctly. 

fftOCUUIlES POft MAINTAININC CONPlDeNTIAUTY 

I. AM AdvMC«4 TtdMoiofy tm(>loyt«» ini ItoM wortttrt ihftU 0^ this 
auuTMCt of conll^Mituyiy. IhU Msuronco niAy bo i>»ortodod by aiioihor 
aiKiranct lor o ^ilcular iwojoci. 

'2. Flow workofft iImII kotp compkttly conlklmlUI tho Mmtt ol rMpomknts, 
oil kilormotion or o^Moiu coUoctod in iho covrso ol kiiorviows, ffid any 
kUormotlon obom rotpondonti looriwl inckkmoUy ^ing lioM work. FtoM 
workoft iholl OMorcUo roMonoblo caution to prevoAt accou by othort lo 
Mirvoy dau 1a Ititk pomsslofu 

I* Uoitu tpoclllcally ifutructed otherwlie for a particular proj«ct» an tm- 
pbyto or llald worlctr, upon cncountorb)* a rtipondont or information 
portakOnc lo a rtipondont Ihtt ho/iht Icnovi p«r$onaily» ^11 ImmtdUttly 
Itrminatt tht activity and contact hl$Aior tMptrvltor lor kdliTuctlons* 

Survey data containing pereonaj idAntlliert In Advanced Tocbnology olllctt 
Shalt b« Iccpt in a Socfctd containor or a lodctd room whtn not btli^ used 
each worlting day in routine survey actlvltlts. Reasonibk caution shall be 
oMercised in Umlting accoas to survey dau to only those pers^yu who are 
working on the specific pro)ect and who have been instructed ki Iho 
applicable confMentlaUty rei^ukements for that pro}«:t 




^kt partloilarly iimflljiNr\kf fM^^^^ 
t or Ikt fffi^;M'^^ 



Where survey 4a ta have boon dttor mined lake | 
Corporato Olficor in €h«rg« ol Iko proioci < 

To ch n ol ogy» tuck sorvoy 4Ma ikaN bt bapi ki lockod^coMokMrt or iii « 
lodtod room osctpi vkiii ocliMlly bokig mod and attondod ky a staff 
Minhor who kM ilgnod iMa pM|«« 

5. Ordkiarily» lorlol WMokta akallko asaigpiod to rttpandtwts prior tocroatii^ 
a mackli^*pfo cott lbt o record ami Mtmii k r a tuck as iUimo« o ddra i a, and 
Mcial tocwrlty OMokor ikftU not»'ardkiirlly» ko a part ol iho modjino 
record. Wh«i idiotlliors oro part ol Iko wacblno dau rocordt Advanced 
T ac t wiala gy * t ii m i g i r ol DMa P rocenki g shall bo rospomiMt lor datonoift' 
Ing id n M i to conftdofttlallly aioatowi In cooii Hi tlon wilk the pr^t 
diro^ior* Wkon a lopwito Mi It sat iijp comiinkig Idonlllora or Ikdcago 
inlormatlo«i ifkkkcoyMko uiad !• fidentily daU rocordt» ikU aoparoto file 
shall bo kopt loehtd up wkon not actually kokig utod oack day in routine 
survey actlvltioft» 

4. Vhon records orilb Idonlillora aro 10 ho trintwdtud to onolkor pirty» suck 
as for keypundOng or key tafinc« tke otkar party #ioU bo kJormod of these 
procedures and shall tiga an Assurance ol Cent Went iakty form. 

7. each pro)oci ^kecior shall ke raspmilblt lor onturb^ tkat all personnel 
and coniroctora Involvod bt kandkn g twrvoy dau on a protect nro kiatructod 
In these procodurtt, ksvo s i gno d tkU p ie ^ e» and comply wiik those 
procoduros l hrq u# >iut tht period of survey performance. Wkon there are 
spocUlc contrackial okligationi in tkt cktm rogard^g conlMontiakty» tke 
pro|oct dkoctor shatt Insintct floM stalls clorkal aiall* conooltantt, and 
any other poroons tdm work en Ikt pre)act ki ttitto additWial procedures. 
At the end ol ikoptrMol aiirvoy ptrlormancot Ikt pro{act director shall 
arrange lor proper tlorago or di t poiltkn ol turvi^ data ksckidk^ any 
particular cantraciual rofoktmonts lor storage or disposition. When 
re^dred to turn over turvoy diu to our cUonts* we must provide proper 
safeguards to ensure conlldonllality to the time lor dellvtry. 
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t. Project dirtctort sh«ll «mure thai turvty pr«ciktt ft^hert to the provi- 
sions of ih* U.S, Privacy Act of 197% with regard to surveys of kufiviikMU 
for ttw Fecteral Govermnent. Pro|cct directors must ensure thbt proce* 
dures ire etiabHslied in eadi survey to inform each respondent of the 
authority for tiie survey* the purpose of the survey, ttw voluntery nsturc of 
the survey (where applicehie), end the effects on the respondents. If eny, of 
nat reipondinf. 



I hereby certify that I have carefully read and will cooperate fully with the 
al>ove procedures, i will Icecp compkteiy confidential «ll Information arlsinf from 
surveys cancerning individual respondentt to which S gain access. I will not discuss, 
disclose, disseminate, or provide accoM lo survey dau and Idcntifiert cacept as 
authorized hy Advanced Tochnotogy. In addition, I will comply with any additional 
procedures eftebiiihod hy Advanced Technology for a pariiculac contract* I will 
devote my heat efforts to ensure that there la compliance with the refuired procedures 
by personnel whom I supervise. I underttand that violation of this pledge ia sufficient 
grounds lor disciplinary action, inchiding dismissal. I also understand that violation of 
th4^ privacy ri^ts ol IndifiduaU throu^ such unauthorized discussion, disclosure, 
dissemiiution, or access may make me subject to aiminal or civil pcnaliies. I give my 
personal pledge that I shall abide by this assurance of confidentiality. 



6-1 ADVANCED TECHNOLOGY'S CONFi: >:NTIAUTY PLEDGE (CONTINUED) 



pteoGC 



Signature 




ASSUBANCE O F CONFIDSNTIALITY QF SURVgY QATA 

Stat^mtnt qf Pftlicv 

Westat i3 firmly committed to the principle that the confidentiality of individual data obtained through 
Westat surveys must be protected. This principle holds whether or not any specific guarantee of 
confidentiality was given at time of Interview (or self-response), or whether or not there are specific 
contractual obligations to the client. When guarantees have been given or contractual obligations 
regaiding confidentiality have been entered into* they may Impose additional requirements which are to be 
adhered to strictly. 

Procedurea for Maintaining ConfldentlaUty 

1 . All Westat employees and field workers shall sign this assurance of confidentiality. This assurance 
may be superseded by another assurance for a particular project 

2. Reld workers shall keep completely confidential the names of respondents, all Information or 
opinions collected in the course of interviews, and any Information about respondents learned incidentally 
during field work* Reld workers shall exercise reasonable caution to prevent access by others to survey 
data in their possession. 

3. Unless specifically Instructed otherwise for a particular project, an emptoyee or fleW worker, upon 
encountering a respondent or information pertaining to a respondent that s/he knows personally, shall 
immediately terminate the activity and contact his/her supervisor for Instructions. 

Pled^ie of Confidentiality 

I hereby certify that I have carefully read and will cooperate fully with the above procedures on 
confidentiality. I will keep completely confidential all Informatton arising from surveys concerning Individual 
respondents to which I gain access. I will not discuss, disclose, disseminate, or piovkie access to survey 
data and identifiers except as authorized by Westat for a partteuiar contract I will devote my best efforts to 
ensure that there is compliance with the required procedures by personnel whom I supervise. I 
understand that violation of this pledge Is suffk:ient grounds for disciplinary actton. Including dismissal. I 
also understand that violation of the privacy rights of Indivkiuals through such unauthorized dlscusston, 
disctosure, dissemination, or access may make me subject to criminal or civil penaKles. I give my personal 
pledge that I shall abide by this assurance of confidentiality. 

SIGNATURE 

I nave road this memorandum. I agree to Westafs Pledge of Confidentiality. 

Reldworker Name; . 

(Print) 



(Signature) 

Social Security No.: 

Address: 



Date: 



EXHIBnr 6-2 WESTATS CONFIDENTIALITY PLEDGE 



r . •'^^^^.'H.v- 



1 

0» 



miL 14, 1906 



IN 


SDOGL NRME 


DATE 


03-12 


1 

GOf ML neUKIDI. SaiNRRV • 1 


1 

2/141 


OI-«7 


SNIINiaUK • 1 


2/171 




IMIVEBITY I 


2/171 




C11V IMV OF m WW-CENTIM. 1 


2/171 


0S-i3 


CflM» COUNTY COLLEGE 1 


2/171 


07-OS 


HMDNE in NDGP Sn OF MJBINB • 1 


2/171 


OHM 


FMCIS NMim miOE 1 


2/171 


OHO 


MUM *m» mmn collek i 


2/171 


10-11 


MUER auiiY amMin mm i 


2/171 


lt-«6 


mCV aUEBE OF KmiT 1 


2/171 


11-07 


NIflMLM PMK amiUTV COUfSE • 1 


2/171 


12-01 


iwMM cnrmi imversitv i 


2/171 


13-07 


own lipscom caiac i 


9/171 
Cf Iff 1 


l(-09 


GOUMIMaUaE • 1 


e/171 


1M3 


NWIfiON iMEA TBHUCH. COLLEGE 1 


2/171 


1^01 


NILOK COLLEK 1 


2/171 


21-0& 


Nil nJIINB COMIMITV COLLEGE 1 


2/171 


2M2 


IM»«6ITY OF COLOMDO • 1 


2/171 


15-10 


NONRIti? WLLEVOMUNlTVCOIiFV 1 


2/111 


20^ 


KM MO,CM IMIVEKIIV 1 


2/111 


23-03 


1 N tmt INSTITUTE 1 


2/lBI 


24-06 


SfM FaPNL!9C0 OnY CL6 CENTERS 1 


2/161 


2S-17 


CVME88 COLLEGE 1 


2/181 


01-12 


NMNMP UNIVEISITV/RABaiFFE COUEGE 1 


2/191 


OS-12 


P 1 HETMDO OF HMR K8IGN 1 


2/191 


01-14 


MERICRN COLLEK FDO flPPLIEI ARTS t | 


2/191 


(»<i 


UnVEMIIY OF FUMIM 1 


2/191 


l<H» 


MNRNBO VRUEY SOOL OF WSINESS 1 


2/191 


11-01 


ORMUM OmiMTV COLLEK • 1 


2/191 


12-oe 


J. HESn LIGHT CRKEN CENTER 1 


2/191 


t(-OI 


UNI^niTY OF NlSBOURl-COLIMIA i 


2/191 


17-03 


KTROIT LAKES MER VDC TECH INST 1 


2/19) 


ll-OB 


UUVEIBITY OF NI9C-NM180N • 1 


2/191 


l»H)4 


NORTH TEIRB STATE IMIVERIilTV 1 


2/191 


22-09 


NENORIAL HDGP SGH OF IMR-TECH 1 


2/191 


25-12 


UNIVERSITY OF OnLIFUMIAHili 1 


2/191 



DC 
N»E 




186 



RECEIPT CONTROL LOG 



SRR IB lATCK 8M 
EIP KC NO REC 



-I- 



IIFF KC COSINB K 
EI/RC DATE DATE DATE 



.| 1. 



■I- 



-I- 



11 II 


i 1 i 


0 i 2/11 


12/20 


1 


7 1 11 


1 • 1 


-1 1 2/20 


12/21 


1 


IS 1 It 


i 11 1 


4 i 2/27 


13/4 


1 


47 1 11 


i 31 i 


1( 1 2/27 


13/S 


1 


10 1 11 


i i i 


4 1 2/20 


12/21 


1 


11 11 


i 2 i 


-1 1 2/19 


12/20 


1 


9 1 11 


i 9 i 


0 i 2/20 


12/24 


1 


2 1 11 


i 2 1 


0 i 2/20 


12/21 


1 


7 1 11 


i 7 i 


0 i 2/20 


12/24 


1 


12 1 11 


i 12 i 


0 1 2/20 


12/24 


1 


fit II 


1 II 1 
1 III 


A 1 9/>A 
V 1 CfCn 


19/9^ 
ICfO 


1 
1 


• 1 1 ) 


i 9 i 


-1 1 2/21 


12/24 


1 


11 1 11 


i 12 i 


-1 1 2/21 


12/24 


1 


• i 11 ' 


i 9 i 


-1 1 2/20 


12/21 


1 


It 1 II 


i 10 i 


S 1 2/24 


12/23 


1 


( i 11 


i S i 


1 1 2/20 


12/21 


1 


II 1 II 


i 9 i 


c 1 cf c4 


Ic/SS 


1 
1 


9 1 11 


i 10 i 


-1 1 2/21 


12/24 


1 


Sill 


i ( 1 


-1 1 2/24 


12/25 


1 


17 i 11 


i 10 i 


7 1 3/21 


1 


1 


12 i li 


i 9 i 


3 1 2/24 


12/28 


1 


10 i 11 


i 7 i 


3 t 2/24 


I2/2S 


1 


Si 1 i 


i 3 i 


2 1 2/2& 


12/26 


1 


7 i 1 i 


t 7 i 


0 1 2/24 


12/26 


1 


Si 1 i 


i 4 i 


1 1 2/24 


12/25 


1 


1 i 1 i 


i 1 i 


0 1 2/24 


12/25 


1 


IS i 1 i 


i IS i 


1 1 2/27 


13/4 


1 


Si 1 i 


i 7 1 


-2 t 2/2S 


12/25 


1 


Si 1 i 


i 7 i 


1 1 2/24 


12/25 


1 


1 i i i 


1 1 i 


0 1 2/21 


12/21 


1 


21 i 1 i 


1 22 i 


-1 1 2/24 


12/25 


1 


Si 1 i 


1 9 i 


-1 1 2/24 


12/25 


1 


12 i 1 i 


1 14 i 


-2 1 2/27 


13/3 


1 


25 i 1 i 


1 23 i 


2 1 2/26 


12/26 


1 


1 i 1 i 


1 1 i 


0 1 2/21 


12/24 


1 


23 i 1 i 


1 SO i 


3 1 2/27 


12/26 


1 



KC an» 

DATE CYCLE 



EXHIBIT 6-3 RECEIPT CONTROL LOG 



1B7 



numbers and filed by batches. The batch system was primarily designed 
for key punching and editing. The manager of coding kept logbooks of aii 
assigned batch numbers. The batchitig system was also incorporated into 
the automated receipt control system for tracking all documents. Thus, 
any given IQ or SRA could easily be located and both its date of receipt 
and progress through the coding and editing cycles could be monitored. 

The Student Data Forms (SDF's) were used by Westat to create the 
initial data files for all sampled students. These files formed the 
basis upon which the Westat field staff could begin to locate the sampled 
students and their parents in order to schedule the interviews. Our 
coding staff checked all SDF's for completeness and correct Social 
Security numbers; any cases with missing or incomplete information were 
referred back to the data collectors for correction. The SDF's were 
photocopied, with the original going to Westat and the copy being 
retained and filed with the appropriate batch at Advanced Technology; 
SDF's were delivered co Westat twice a week. 

6.2.2 Student and Parent Data 

Receipt control, using computer-generated logs of all materials 
acquired relating to the student and parent sample, was the monitoring 
point for all student, parent, and secondary data collection materials 
sent to the field and returned to Westat. The master receipt control log 
listed the study identification number, name, address, and Social 
Security number of each student and his or her parent(s). The log was 
organized in numeric order according to study identification numbers. 
Space was provided for recording updated information on names and 



addresses, the date each questionnaire was received from the field, the 
completion status of the document, and the coding batch number assigned 
to the document- Within the record fo^ each pair of student and 
parent(s), separate entries were made for the receipt of the student and 
parent questionnaires or other disposition of each member of the pair- 

6«2*3 Secondary Data 

' A separate receipt control procedure was used for the secondary 
data- For two of the three secondary data sources (IRS returns and 
financial institution records), the receipt control procedures included 
first logging in the release form signed by the student or parent, 
checking the form for accuracy, and then sending the release form to the 
appropriate financial institution or IRS Service Ceccer- Later, when the 
request was returned from either of these two sources, the returning 
documents were logged in, assigned to a coding batch, and sent for coding 
and editing- The collection of tax assessment data to verify home value 
did not require a release form to be signed by the respondent, which 
permitted the omission of the first receipt entry. Separate columns on 
the master log were also used to keep track of the secondary data for 
each student and parent in the sample. 

6.3 DATA PHEPARATION 

After data collection fctms were received at the Advanced Technology 
or Westat offices, they were thoroughly edited for completeness and 
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consistency before being sent to keypunch for conversion to tape. Forms 
from che inscitutionai visits--IQ*s and SSA's— were coded and sditsd at 
Advanced Technology. Student and parent interview forms and secondary 
data collection forms — IRS data, financial institution data, and tax 
assessor data — were coded and edited at Westat. 



6.3.1 Coding and Editing Staff 



A well-qualified group of temporary coders/editors was selected for 
this project. Of the six individuals on the coding/editing staff, one 
had coding experience with the Stage One data collection/ and two others 
also had similar experience outside the firm. Five of the six 
coder/editors were college graduates, and one had an advanced degree. 

Coders were hired and trained at the beginning of the institutional 
data collection so that coding could take place simultaneously with the 
data collection/ as forms were received from the field. The 
ceding/editing supervisor briefed them on the project/ the financial aid 
programs involved, and the field activities. Tlie study objectives were 
emphasized in training/ reinforcing the importance of this understanding 
as contributing to the coders* ability to accurately categorize and 
interpret survey responses. This also gave the coders the opportunity to 
recognize problems or errors and, with the coding supervisor/ resolve 
many discrepancies. They reviewed the coding manual and the 
Question-by-Question Specifications before a discussion with th3 
supervisor of coding procedures and conventions. The manager for 
analysis also briefed them on important points. The supervisor reviewed 
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in detail the first instrumGnt completed by each coder and discussed all 
errors with the group. She then reviewed iOO percent of the totms coded/ 
until the error rate declined to less than 1 percent for each coder/ 
editor. Thereafter/ coded instruments were reviewed on a sample basis; 
review of problem cases also provided opportunities for review of coder 
performance. 

6 •3,2 Coding the Student Record Abstracts 

The coding and editing supervisor developed a coding manual which 
included general coding procedures to be used across the entire 
instrument and special procedures for certain items. Coders also were 
provided copies of Question-by-Ques tion Specifications / which contained 
additional information about each item. For most items / the codes were 
already included in the instrument. Some items had a restricted list of 
answers. For other items / the form had preprinted codes for the most 
likely answers/ based on the responses to similar items in the earlier 
quality control studies. Project analysts developed lists of codes for 
other dJASwers. 

The coders/editors were directed to refer questionable items to the 
coding and editing supervisor. All items with recorded responses which 
had no code in either the form or the codebook were referred to the 
supervisor. Many items included provisions for explanation of 
institution actions/ decisions/ or documentation failures that might 
violate regulations and/ therefore^ result in errors; all of these cases 
were also referred to the coding supervisor. The responses were 
evaluated in all of these cases / often in consultation with other project 
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analysts, to either resolve the problem or develop new, more appropriate 
codes* 

In addition, the Institutional Questionnaire for each institution was 
coded at the same time as the SRA's. This proved to be an efficient use 
of tlie coding staff, particularly since it gave them a familiarity with 
'-^titutions that was often helpful in coding the SRA*s. 

The largest class of items referred by the coder/editors was 
comprised of inconsistencies and missing information. In a small 
percentage of cases, these could not be resolved by the project staff. 
As discussed in Chapter 3, we were able to resolve these cases through 
contacts with the data collectors (during the field perio*5) and the 
institution (after data collection). We received uniformly excellent 
cooperation from institutions in filling in missing data, resolving 
apparent contradictions, and obtaining additional explanations beyond* the 
notes made by the original data collector. 

To facilitate tracking the docxxments, a.\l the instrximents from one 
institution were kept together throughout the coding process. All were 
handled by the same coder/editor, and if one had to be referred to the 
supervisor, all were. Only when all an institution's forms were 
completed, with all problems resolved, were they assigned to a batch for 
data entry. 
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6.3.3 Coding the Institutional Questionnaire 

The principal difference between coding the IQ*s and the SRA*s was 
the presence of open-ended questions on the IQ*s. The preliminary coding 
scheme was based on that used for the Stage One instrument, and was 
supplementev^ by the creation of new codes and categories of codes for the 
Stage Two instrument. Since many items on the IQ were interrelated, the 
answer to one open-ended question often provided information about 
another. Coding the IQ's required a careful reading of each instrument, 
and a large number of codes to portray accurately the range of responses 
given. 

All open-ended responses and all other responses about which the 
coders had questions were referred to the coding supervisor. Every 
unique response was given its own initial code. Later rounds of coding 
involving problem responses and some of the open-ended items were handled 
by project analysts. After all the instru:.ients had been coded these 
codes were reviewed by project analysts and grouped into more 
comprehensive categories. 

6. 3^4 Preparation of Student and Parent Data 

Westat prepared a coding manual incorporating both the student and 
parent questionnaires used in the Title IV Quality Control Project. The 
coding manual consisted of an introduction to the study procedures and 
purposes, a review of general data preparation procedures, and coding and 
editing specifications for both data sets. The manual was used in 
training the data preparation staff, and served as a complete, detailed 
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reference for all project staff. It also provided dociimentation for the 
Title IV study data files. 

Ten survey processing personnel were selected for training as coding 
and editing staff for the student and parent data portion of the Title IV 
study. Two were assigned to be group leaders^ based on their skills and 
qualifications. The group leaders served as assistants to the coding 
supervisors and as coder verifiers. 

Coders were assigned to work by coding batches and were required to 
complete the coding of one batch before beginning work on another. 
Errors found during verification by the supervisor were first noted in a 
coding error log, then discussed with the coders responsible for them. 
If persistent errors were discovered, a coder was asked to review 
previous batches and correct them. Problems found during coding, but not 
resolved in the coding specifications, were documented and referred to a 
supervisor to be resolved. Particularly difficult cases were referred to 
a weekly meeting of senior Advanced Technology and Westat project staff 
for resolution. 

The Student and Parent Questionnaires included several open-ended 
questions. It was not possible prior to the beginning of coding to 
device lists of all the possible responses to these items. Since this 
problem was expected, a controlled system for dealing with it was 
implemented at the beginning of coding. Responses whi'.h were not codable 
in the predetermined list of codes from the coding specifications were 
docuinented and referred to the supervisors who constructed codes for the 
new items. New codes were published each morning on a coding change 
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sheet. Coders were responsible for keeping their manuals up to ^ 
were required to record each issue of the coding changes in a log. 



6.3.5 Coding the Secondary Data 

The same procedures that were used to code the student and parent 
data sets were also used for the IRS returns, the financial institution 
records, and the tax assessment records. A separate manual was produced 
for each. 

Occasional problems with illegible figures arose in the coding of 
photocopies of IRS tax forms. It was sometimes necessary in these 
situations to code illegible data elements as missing values. In 
addition, some taxpayers do not completely fill out Form 1040A- when 
filing, exercising their option to have the IRS calculate their taxes. 
These 1040A*s are blank below line 4. Westat coders were trained to fill 
in the missing items on these blank for.Tis using a 1984 tax table. 

6.4 DATA EHTHY 

Once all the data collection instrui.ients were coded and edited, the 
coded data were entered i-^o an automated filing system, resulting in a 
set of data tapes that included all data from all sources. This was the 
first step in the creation of the master file that would be used for our 
analysei. 
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6.4.1 Institutional Data 



Key entry of Advanced Technology's questionnaire data, the SRA and 
the IQ, was performed by a local data entry vendor. To ensure the safety 
and confidentiality of the data, the vendor's in-house courier 
transported all documents and keyed data tapes to and from Advanced 
Technology. 

The manager of data processing thoroughly reviewed the questionnaires 
with the data entry supervisor before any data were keyod. During this 
meeting, he clarified questions regarding the layout and design of the 
questionnaires, the interpretation of data fields, and notation used by 
coders. He also provided a detailed set of specifications for the 
physical layouts of the data tapes the vendor would be creating (i.e., 
.record length, blocking information, labeling, and tape density). 

All data were keyed and then 100 percent key-verified. This process 
required the data to first be entered by one operator and then keyed 
again by a different operator. Any discrepancies between the two were 
noted on a computer screen and then resolved. To assure further the 
accuracy of the keyed data, we performed an in-house review of selected 
cases, checking the actual documents against the keyed data file. the 
results of this review showed the data entry to be over 99.5 percent 
accurate. 

6.4,2 Student/ Parents and Secondary Data 

Data entry was performed on all student, parent, and secondary data 
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instruments by Westat Data Entry Services (WDES), a support group located 
in Westafs Rockville office. The system used is a Key-Edit Model 2022 
disk data entry system. 

To ensure accuracy, all keypunching of data instruments for the 
student and parent survey and the secondary data forms were verified by a 
double entry procedure. With this method, the total data entry error 
rate did not exceed 0.5 percent of the total number of strokes keyed. 

Each data set was keyed to disk, then transmitted to tape, so that 
the machine edit procedures could be carried out. The final steps in 
creating the clean data file involved first producing frequency 
distributions of all variables so that final edit checks could be 
conducted, then generating the deliverable tapes. 

6,5 COMPUTER EDITING 

The next step in creating a clean analysis file involved subjecting 
the keypunched data to an automated editing process. These edits 
consisted primarily of range checks to ensure that the values of all 
variables were within the maximum possible or likely ranges, logic checks 
to determine whether there were any unreasonable relationshsips between 
variai:)les (such as people recorded as not having filed a Fc*< al income 
tax return, but having paid income taxes), and checks that skip patterns 
within the instruments had been followed properly. 
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6*5.1 Institutional Data 



Computer editing of data from the SRA and IQ consisted of checking 
the coded data for range and logic errors. We checked each distinct data 
field to make sure that only valid codes or acceptable values were 
present. Logic checks for both the SRA and the IQ were developed by the 
manager of data processing and reviewed by another project analyst. One 
set of logic checks was developed for each questionnaire to ch^cjc *or 
correctness and to determine whether data collectors and coders l.^ad 
followed the skip patterns correctly. A second set of logic tests was 
developed to test the internal consistency of the data on each 
questionnaire. Responses that seemed contradictory with other 
information on the questionnaire were flagged in the logic check section 
and reproduced in the edit report. 

Final range and logic tests were then translated into SAS cod^ and 
incor^ 'ated into edit programs. We developed two programs, one for the 
SRA and another for the IQ. After development and testing of these 
programs, an initial run was made against the raw data tapes. At the 
completion of oach run of the edit program, an edit report was produced. 
Each edit report included counts of the nxxmber of cases edited and the 
number of cases with detected errors. Any case identified a& having 
either range or logic errors was reproduced in the edit report. 

All edit reports were passed on to the editing staff for error 
resolution, through comparison with the original questionnaire. All 
errors not easily resolved by the editors were referred to the supervisor 
for review. File update transactions which would rectify the items in 




error were prepared by the data editors and entered into a SUPERWYLBUR 
transaction file. The transaction file was first processed through an 
on-line macroprogram which checked the keyed transactions for syntax 
problems. If no syntax problems were discovered in the transaction file^ 
the macroprogram called an update program which posted each item in the 
transaction file against the master record. The edit-update cycle also 
included a final quality control check on the data entry process / to 
assure an accuracy rate of 99.5 percent. This process of editing and 
updating data was repeated until edit reports showed no unexplained or 
unacceptable errors and a batch could be considered "clean." The 
edit-update cycle is shown in Exhibit 6-4. 

6.5.2 Studen*- and Parent and IRS Data 

The student. parent/ and IRS files were machine edited with 
special-purpose editing programs generated through Westat*s Codebook and 
Edit (COED) system. Specifications for the edits were produced by Westat 
and reviewed and approved by Advanced Technology. Edits concentrated on: 

• Valid data in numeric fields 

• Valid range checks 

• Correctness of skip patterns 

• Consistency of response patterns 

• Special edit logic as required. 

Theso edits were coded, reviewed, and converted into edit programs 
for use by the Westat pr^a^ct staff. The coding supervisor was also 
responsible for machine editing and was familiar with all aspects of 
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the instruments and the edit facility. The machine editing staff were 
trained coders for the Title IV project, so that they were already 
familiar with the forms r 

Edit runs were initiated using a programmable command procedure 
designed for this project. Machine editors received listings with all 
errors clearly marked and resolved errors according to instructions. 
Initial edits were completely reviewed by the edit supervisor to ensure 
that the machine editor's work was completely correct. Difficult cases 
were referred to the coding supervisor or resolved in a weekly meeting of 
senior project staff. 

File updating instructions were written on transcription sheets by 
the editors/ checked by the supervisor, an^. len sent to the data entry 
center for keying and transmittal to the main computer. Updates were 
made to the files by a special-purpose update program. The jobs were run 
in a manner similar to the initial edit runs. After each update run was 
complete, another edit cycle was initiated automatically to verify that 
corrections had been made and to check for new errors. The update cycle 
was repeater', until each batch of data in the data base was clean. 



6»5.3 Secondary Data 



Different procedures for secondary data were adopted due to the small 
number of items on the forms and the small number of forms received. 
These data include the financial institution records and tne tax 
assessment records. These files contained about six fields and only 
about 100 records each so that a manual edit was more efficient than the 
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construction of an automated edit program. Experienced coders reviewed 
the data tor numeric and logic checks both before and after key entry. 
This approach actually provided for a more individualized review than 
would be possible on the larger forms* 

As a final quality control check, marginal distributions of all 
variables on files were produced and reviewed prior to delivery to 
Advanced Technology, No unanticipated problems occurred during this 
phase* 

6.6 MERGING OF DATA SETS 

The error calculations for Stage Two required data from all of the 
files created through the data preparation procedures. Therefore^ we had 
to create a merged file containing in a single record all the variables 
from the seven different files for each student. Each file consisted of 
the data from a single source; 

♦ Student Questionnaire (SQ) 
0 Parent Questionnaire (PQ) 

♦ Income tax returns from IRvi 

♦ Financial institution records (FIR) 

♦ Tax assessment records (TAH) 

♦ Pell Grant processor Computed Applicant Record (CAR) file 

♦ Student Record Abstract (SRA). 

The first five of these files were created by Westat. The data from 
the CAR filo vers extracted from a tape supplied by the Pell Grant 
processor by matching the list of Social Security numbers of sampled 



students against the Social Security numbers on the file. The SRA file 
was created by Advanced Technology. The result o£ the merge process vas 
a single analysis file of student data/ as shown in Exhibit 6-5. The 
master analysis file included the student data file and a separate 
analysis file of institutional data, obtained from the Institutional 
Questionnaire and containing no individual-level data. 

6.6.1 Merge Programs 

Aft "^r each of the seven primary student data sets had gone through 
the entire edit and update process, we created an analysis file 
containing data elements from each data set. This merged file was 
developed through a series of data merges using the SAS file-combining 
capabilities (see Exhibit 6-5). The process was divided into four basic 
steps. First, four of the files produced by wescat {?Q, SQ, FIR and lAR) 
were combined into a single file. Second, the SRA and IQ data were 
merged and then combined with the CAR data creating a single file of 
Advanced Technology data. Tnird, the combined Westat file was merged 
with the Advanced Technology data into a single file. Finally, IRS data 
was merged with the combined Wes tat/Advanced Technology file creating a 
single analysis filv^. 

We also developed several smaller programs to make final changes in 
the structure and makeup of the primary data sets immediately before 
merging that would facilitate the merge process. These final 
preparations included reformatting the IRS file, sorting the files, and 
transferring files from tape to disk storage. 
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EXHIBIT 6-5 MERGING OF STUDENT DATA 
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6.6.2 Selection of Best Values 



The term, "best values/' was introduced in the Pell Grant Quality 
Control Study tc represent a specific aspect in the measurement of 
error. Error is based on the difference between reported values - those 
reported on the application or adjusted by the inscitution (for student 
values) and those used by the institution (for institution values) - and 
"best/" or verified/ values from other data sources obtained during data 
collection. The determination of best values involved a complex 
comparison of data items that considered not only the source of a value 
for a given item (that is, where in the data collection it was obtained)/ 
but also the strengths or priority, of the documentation for that value. 
For example, for best institution values, we specifically tailored some 
questions in the Student Record Abstract to allow us to determine best 
values while other questions simply recorded the values used. 

An example of how we used the SRA to determine institutional best 
values is in cost of attendance for the Campus-Based programs. After 
noting the figure used by the school, the data collector referred to the 
school's cost of attendance policy. If the amount was correct, the data 
collector indicated as much. If the amount differed from what the policy 
said it should have been, but reflected an individual adjustment, as 
allowed by progra-n regulations, the data collector noted it, as well as 
the information which documented the adjustment, as required by the 
regulations • If the amount used for cost of attendence was not correct 
according to policy, and the data collector found no docxxmentation of an 
adjustment, then the response to this item was "not correct." The data 
collector then attempted to resolve the difference by asking the 
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financial aid administrator to explain the circxxmstances of the case, 
again noting either the explanation or the lack of resolution, and 
recorded what the cost should have been, according to the school's policy. 

For student application items, we developed priorities to 
consistently select the best value from the several competing data 
sources (Student Questionnaire, Parent Questionnaire, Student Record 
Abstract documentation, and three external sources of docximentation, 
including the IRS). Although we developed best value priorities 
expressly for each application item, general guidelines were followed for 
checking the supporting documentation for any values reported. The 
general guidelines for best value selection were: 



• The strongest priority was assigned to values with external 
sources of documentation: 

— For items which could be documented by tax returns, this 
was always an IRS-provided copy. 

— For savings and home value, these were financial 
institution records and tax assessor records, 
raspectively. 

• If external documentation was not available, or possible, 
selected documentation from the PQ or SQ was strongest, 
followed by similar documentation from the SRA* 

• Distinctions were made concerning the strength of 
documentation within a data source. For example^ certified 
tax returns shown by the parent to the interviewer were 
stronger than copies or worksheets. 

• Within the same data source, documentation that was 
considered more reliable or more likely to be complete was 
assigned a stronger priority* Thus, a letter from a relevant 
agency about a parent's other income received was assigned a 
stronger priority than that parent's records. 



In the absence of documentation, we accepted application values as 
best values^ except in two situations. If the best dependency status 
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differed from the reported dependency status; the application values were 
irrelevant because they were for the wrong party (usually the student:, 
when parenc data was needed). In this case, undocumented values were 
accepted as best values if no other data were available, since they were 
parallel to the undocumented valu-^^ reported on an application. 

When the application requested only composite daca. as it did for 
other nontaxable income and benefits, each of the parts could be 
documented but application data were not available for the individual 
components. Thus, if no other data were available for any one item, 
undocumented values were accepted in order to avoid ignoring a source of 
nontaxable income. The total of all the components was then compared to 
the application for final best value selection and used when it was 
greater than the application tJtal. 

If documentation was not requested (as for some of the dependency 

status questions in the student and parent questionnaire), agreement 

between undocumented values from two or more sources was considered 
suitable for a strong priority. 

An example of this system of priorities would be that, for a 
student *s adjusted gross income (AGI). n copy of a student's tax return, 
collected from the IRS, would be assigned a higher priority than a copy 
of the same document seen in the student's financial aid file. However, 
the copy of the tax return seon iu the student* s file would be stronger 
documentation of AGI than a worksheet shown by the student to a Westat 
interviewer. The best value priorities for this item are shown in 
Exhibit 6-6. 
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specifications from the best value priorities were translated into 
SkS code for the con»putatics of best values. Four separate programs were 
required/ the first of which was the determination of a student's best 
dependency status. Cases for which dependency status was determined as 
independent were run through a program which computed best values for 
variables unique to independent students • Cases determined to be 
dependent students were run through the remaining two programs, which 
computed best values for variables unique to parents of dependent 
students, and best values unique to dependent students, respectively. 
Exhibit 6-7 shows the process of selecting best values through the four 
best value programs, and the resulting file of best values for all cases. 

We selected 45 cases at random for hand checking. For each variable 
the computed best value from all possible sources was printed. These 
values were compared with values on the original questionnaires. Best 
values were calculated by hand, following the best value priorities. 
These hand-calculated best values were compared to computed best values. 
The best value programs were modified and the output rechecked until best 
values were correct for the sampled cases. Outputs from the four best 
value programs were then merged together. The resulting data file was 
run through an error calculation program, and 75 high-error cases were 
hand checked. The best value programs were modified again and the output 
rechecked until no error was found in the selection of best values for 
the high->error cases, and the complete data file could be rerun through 
the best values programs. The final merged best value file was used in 
the analysis of award errors. 
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6,7 ANALYSIS PRCX5RAMS 

The analysis for Stage Two involved measuring errors througn a 
comparison of th^i values used by institutions in calculating need and 
awards/ and best values^ as described above. The programs used to 
conduct these analyses were carefully tested to ensure that the 
algorithms were up-to-date and accurate. This required a careful review 
of the need analysis formulae, including the Pell processor, and a review 
of currently applicable regulations. 

An exception to this review and revisions process occurred when 
regulations affecting these calculations were issued and became effective 
during the award year under study^ but after the end of our data 
collection. We were therefore unable to collect the data that would 
allow us to accommodate the change in our algorithm. 

An example of this occurance was the publication of changes required 
by the Consolidated Budget Reconciliation Act of 1985 (Public Law 
99.272). A new regulation resulting from this Act affected the 
calculation of a student's expected family contribution for the Pell 
Grant and Guaranteed Student Loan programs, that **income realized from 
the proceeds of a sale of farm or business assets if ths sale results 
from a voluntary or involuntary foreclosure/ forefeiture, or bankruptcy" 
should be excluded from family income (34 CFR 682.301 and 34 CFR 690.3 3 
and 690.43). This regulation, while applicable to the 1985-1986 ciward 
year, was issued at the end of the award year, and 2 months after the end 
of our institutional data collection. We were therefore unable to ask in 
our instruments if any sampled students may have been affected by this 
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change, nor did we revise our analysis programs. However, we estimated 
that this new regulation would have affected less than one tenth of 1 
percent of Pell applicants, making it highly unlikely that such cases 
would have been selected in our sample. 

6.8 QUALITY COITROL OF COMPUTER PROGRAMS 

All computer programs used in this project were thoroughly reviewed 
and tested before use, as shown in Exhibit 6-8. The tests were run on a 
sample of real data so that we could be sure that the programs would 
treat correctly the actual problems found in the student population from 
which we had drawn our sample. 

6.8.1 Advanced Technology Programs 

The overriding concern of any data processing effo"*" must be assuring 
the quality of its software. Having processed data in a similar format 
during the preceding quality control studies, our data processing staff 
was well aware of the types of problems that we would face in processing 
data for the current study. As a result, the primary quality control 
concern for Stage Two was focused on areas that had previously proven to 
be highly prone to software failures* Some of the programs that had been 
produced for the earlier studies were adequate for use with only minor 
modifications* 

Specifically, the edit programs that had been developed for the 
earlier studies proved to be readily adaptable to the needs of Stage 
Two. The overall structure of these programs was maintained and the 
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range/ logic, and data definition sections were updated to accommodate 
the current data. The output sections were upgraded to provide more 
informative and more understandable edit reports/ internal docxunentation 
was expanded/ and the syntax of the code was modified to improve 
readability. 

As in the past, we tested the programs thoroughly by first applying 
them to a batch of hand-calculated test data. We compared the resulting 
output to hand-calculated results to verify accuracy. The second step in 
testing the programs was to run them on a 10 percent sample of ''live'* 
data. This test performed an additional check on the program to see that 
plausible results were obtained (i.e., whether the resulting analysis 
fell vrithin pre-established acceptable limits). In each of these test 
steps / if errors were detected, they were corrected promptly and 
re-tested until the program was judged acceptable. 

Some quality control checks were performed on ad hoc analyses. Such 
analyses usually required quick response and received a technical review 
by the manager of data processing and the project manager. These reviews 
were less formal than the reviews perforined on the edit or merge 
programs, but no less thorough. They examined required inputs, desired 
outputs, and necessary manipulation of data. Before implementation, all 
ad hoc programs were also put through the same testing procedures used on 
production programs. 

All SAS procedures were also documented internally and externally 
throughout the process. External dociimentation consisted of a detailed 
description of the merge cycle to be recorded on a form, one copy being 



bound with the computer output, the other copy in a looseleaf notebook. 
This documentation described in detail any problems encountered and 
provided the foundation of the final report detailing the merge process. 
The internal documentation consisted of a brief description of the 
inputs, outputs, and process of each SAS program and appeared at the top 
of the source listing as a section of comments. 

6,8«2 Westat Programs 

Edit and merge programs were available to Westat from Stage One . 
Because the data from instruments for the student and parent interviews 
had needed only minor revisions, Westat did not have to write new 
programs, and no substantive new coding was done for the spring 1986 
survey. These programs also had passed the most important quality 
control criterion, successful operation with a wide variety of input data. 

Westat had a series of standard operating procedures for program 
creation, which had been followed in writing the original programs and 
were applied to the 1986 updates. These steps included the following: 

• Construction of a working flowchart with each program module 

• Addition of record counts and file names as i^ach file was 
created 

• Review of each program by a senior systems analyst 

• Binding of program listings with a final flowchart. 

These steps provided an audit trail for all file creation activities. 

The quality of the editing programs was assured by use of the COED 



6-36 

197 



editing system. This system automatically generated computer code from a 
file of machine-readable edit specif j >ationS/ eliminating the possibility 
of programmer error. The coding supervisor specified the edits on the 
basis of experience with the earlier quality control surveys. Senior 
project staff also reviewed the specifications and forwarded them to 
Advanced Technology for final approval. The pTogram generator edited the 
specifications to ensure that no incorrect code was generated. 
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EFFECTS OF NONRESPONSE AND SAMPLING ERROR 



This study is based on a sample of recipients of Pell awards, 
Campus-Based aid or GSL certifications; therefore the values reported can 
only be estimates of the values that would have been obtained ty 
successfully collecting information about every recipient. Moreover, 
even within the small sample we selected, we were not able to obtain 
universal cooperation; best values, then, were unobtainable for some 
sampled students or their parents. In this chapter we analyze the 
effects of this nonresponse and estimate the probable errors in our 
estimates due to sampling. 

7.1 EFFECTS OF NONRESPONSE 

We can analyze the effects of nonresponse in two ways. First, we can 
compare the average respondent to the average sampled aonrespondent for 
characteristics for which we have information on both groups. Second, we 
can determine what our estimates of program-wide parameters would have 
been if the average nonrespondent had been similar to a small group of 
respondents with r^typical characteristics. 

7.1.1 Comparison of Respondents and Nonrespondents 

We examined respondents and nonrespondents for differences on four 
key items which have major effects on need and payment error: 
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♦ Dependency status (all Title IV recipients) 

• Student Aid Index (Pell Recipients) or Expected Family 
Contribuvion (Campus-Base^, recipients) 

• Need (Campus-Based recipients) or amount of GSL eligibility 
(GSL certifications) 

♦ Awards (Pell and Campus-Based recipients) 

All comparisons are based on reported data recorded on SRA's. We usually 
had no docximentation for nonrespondents and/ by definition, no strong 
interview data. Therefore, all comparisons are of reported rather than 
best values. We report separate comparisons of SAI, EFC, need or GSL 
eligibility, anu Pell or Campus-Based disbursements for independent and 
dependent students because the financial situations of the two groups are 
so different. 

As shown in Table 7-1, response rates did not differ significantly by 
dependency status for Pell or GSL recipients. Among Campus-Based 
recipients, dependent students had a higher rate of nonresponse but the 
difference is only marginally significant. 

Table 7-2 summarizes average SAX for respondents and nonrespondents 
by dependency status. Among independent Pell recipients, respondents had 
a higher mean SAI than nonrespondents, but the difference is not 
significant. Among dependent Pell recipients, respondents had a 
significantly higher mean SAI than nonrespondents. 
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TABLE 7-1. DEPENDENCY STATUS OF RESPONDENTS AND NONRESPONDBNTS 



Pell Recipients 



Reported Dependency Respondents 

Status Niunber Percent 



Nonrespondents 
Number Percent 



Total 



Jumber Percent 



Independent 

Dependent 

Total 

x2 = 0.034. df 



851 
1118 
1969 
1. p = 0.85 



87.0 
87.3 
87.2 



127 
163 
290 



13.0 
12.7 
12.8 



978 
1281 
2259 



100.0 
100.0 
100.0 



Campus-Based Recipients 



Reported Dependency Respondents Nonrespondents 
Status Number Percent Number Percent 



Total 



Number Percent 



Independent 

Dependent 

Total 

x2 = 3.699. df 



571 85.6 

1004 82.2 

1575 83.4 
1, p = 0.055 



96 14.4 667 100.0 

218 17.8 1222 100.0 

314 16.6 1889 100.0 



6SL Certifications 



Reported Dependency Respondents 
Status Number Percent 



Nonrespondents 
Number Percent 



Total 



Number Percent 



Independent 

Dependent 

Total 

X2 = 1.115. df 



555 
812 
1367 
1. p = 0.29 



82.8 
80.8 
81.6 



115 
193 
308 



17.2 
19.2 
18.4 



670 
1005 
1675 



100.0 
100.0 
100.0 
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Table 7-2. MEAN STODEMT AID INDEX OF RESPONDENTS AND 
NONRESPCXiDENTS BY DEPENDENCY STATUS FOR PELL RECIPIENTS 



Reported Dependency Mean SAI 

Status Respondents Nonrespondents 



All Cases 



Independent 
Dependent* 
Ali cases* 



249.6 
599.5 
448.5 



227 .2 
395.3 
321.3 



246.7 
573.7 
432.2 



-0.4792 
-4.5926 
-3.8707 



0.63 

0.0001 

0.0001 



TABLE 7-3. MEAN EXPECTED FAMILY CONTHIBUTKW OF 
RESPONDENTS AND NONRESPONDENTS BY DEPENDENCY STATUS 
FOR CAMPUS-BASED RECIPIENTS 



Reported Dependency Mean EFC 

Status Respondents Nonrespondents All Cases t 



Independent 
Dependent* 
All cases* 



2845.3 
1658.5 
2089.0 



2911.9 
2683.0 
2753.7 



2854.9 
1839.4 
2198.7 



0.2276 
6.0536 
4.2850 



0.82 

0.0001 

0.0001 



TABLE 7-4. MEAN NEED FOR RESPONDENTS AND 
NONRESPONDENTS BY DEPENDENCY STATUS 
FOR CAMPUS-BASED RECIPIENTS 



Reported Dependency Mean Need 

S tatus Respondents Nonrespondents All Cases 



Independent* 
Dependent 
All cases* 



6303.3 
5683.2 
5908.6 



7295.9 
5722.1 
6206.3 



6445.9 
5690.1 
5957.8 



1.9782 
0.1611 
1.2782 



0.0503 

0.87 

0.2019 



* Variances are significantly different 
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Table 7-3 sxxrranarizes mean EFC for respondents and nonrespondents by 
dependency status. Among independent C-B recipients respondents had a 
smaller mean EFC than nonrespondents but the difference is not 
significant* Among dependent C-B recipients the mean EFC is 
significantly higher for nourespondents than for respondents. 

Table 7-4 presents mean Campus -Based need for respondents and 
nonrespondents by dependency status. Among independent C-B recipients, 
average need is higher for nonrespondents than for respondents. 
Nonrespondents al.so have a higher average need than respondents among 
dependent students, but the difference is not significant. 

Table 7-5 summarizes average GSL certifications for respondents and 
nonrespondents by dependency status. Among independent students with GSL 
certification, the average certification is higher for nonrespondents 
than for respondents. The difference is not significant. Among 
dependent students, the average certification is significantly higher for 
nonrespondents than for respondents. 

Table 7-6 summarizes average Pell awards for respondents and 
nonrespondents by dependency status. Among independent Pell recipients, 
respondents have a higher average Pell award than nonrespondents. The 
opposite is true for dependents. Neither of these differences is 
significant. 

Table 7-7 summarizes mean Campus-Based awards for respondents and 
nonrespondents by dependency status. Among independent C-B recipients, 
nonrespondents have a higher mean C-B award than respondents. The 
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TABLE 7-5. MEAN GSL CBRTIFICATI(» FOR RESP(»IDENTS 
AND MONRESPCMDENTS BY DEPENDENCT STATUS 



Reported Dependency Mean GSL Certification 

Status Respondents Honrespondents All Cases t 



Independent* 
Dependent* 
All cases* 



2256.3 
1974.3 
2087.5 



2416.2 
2173.9 
2264.7 



2283.6 
2012.7 
2120.1 



1.3767 
2.517 4 
2.6786 



0.1707 
0.0124 
0.0077 



TABLE 7-6. MEAN PELL AWARD FOR RESPONDEMTS AND 
HONRESPONDENTS BY DEPENDENCY STATJS 



Reported Dependency Mean Pell Awar d 

Status Respondents Nonrespondents All Cases t 



Independent 
Dependent 
All cases 



136a. 3 
1242.9 
1297.1 



1265.1 
1303.5 
1286.1 



1355.0 
1250.6 
1295.7 



-1.7746 
1.2363 
-0.2916 



0.08 
0.22 
0.77 



TABLE 7-7* MEAN CAMPUS-BASED AWARD FOR RESP(m)ENTS AND 
NONRESPONDENTS BY DEPENDENCY STATUS 



Reported Depend, ncy Mean Campus -Bas^d Award 

Status Respondents Nonrei^pondents All Cases t 



Independent 
Dependent 
All cases 



1423.7 
1465.6 
1450.4 



1526.4 
1436.1 
1463.6 



1438.4 
1460.3 
1452.6 



0.8688 
-0.3930 
0.2090 



0.39 
0.69 
0.83 



* Variances are significantly different 
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opposite is true among dependents. Neither of these differences is 
significant. 

In sununary, the most significant measures are SAI, E*'^'', and amount of 
GSL certification. Table 7-8 summarizes the relationships between these 
measures and the incidence of error in the respective programs. For the 
Pell program, mean SAI is given for groups defined by direction of 
overall Pell error (under, over, none). For the Campus-Based programs, 
mean EFC is given for groups defined by the direction of need error 
(under, over, none) and by the incidence of awards in excess of need 
(over and none). For the GSL program, mear GSL certification is given 
for groups defined by the incidence of overall GSL certification error 
(over and none). The level of significance is given for each comparison. 

Since nonrespondents have significantly lower SAI*s than respondents, 
and lower SAI's are associated with no error, the incidence of error may 
have been overestimated. The likely effect on the magnitude of error is 
not easily projected because higher SAI*s are associated with both under 
and over awards. 

Nonrespondents have significantly higher EFC*s than respondents. 
Higher EFC*s are associated with koth need error and awards in excess of 
need. Thus, the incidence of these Campus-Based errors may have ^een 
underestimated. 



Significant 
Konresponse 
Keasure 



TABLE 7-8 • SIGMIFICAN^r NCmESPONSE MEASURES BY 
IHCIDCi^CE AND DIRSCTICIi OF ERROR 



Means 



Direction of Error 



Error 



Under 



None 



Over Significance 



SAI 

EFC 
EFC 

GSL Certification 



Overall Pell Error 702 288 671 0.0001 

Overall Need Error 2210 1367 2433 0,0001 

Overall Awards in 

Excess of Need N/A 1985 2541 0,0001 

Certification Error N/A 2408 2239 0,0004 
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Nonrespondents have significantly higher GSL certifications than 
respondents* Higher GSL certifications are associated with nc 
certification error. Thus the incidence of GSL certification error may 
have been overestimated* 

The incidence of Pell and GSL error may have been overestimated and 
the incidence of Campus-Based errors may have been underestimated. The 
effect on the magnitude of error is not easily determined using this type 
of analysis. The sensitivity analysis which follows describes how non- 
response might affect estimates of total error under extreme conditions. 

7.1.2 Sensitivity Analj^sis 

Comparing /s^spondents and nonrespondents is one way to assess the 
potential effects of nonresponse bias on our error estimates. Another is 
to make various bad-case assumptions about nonrespondents. The analysis 
in Volume 1/ Findings / assumes that the average nonrespondent is like the 
average respondent v pa that, therefore, average errors would be unchanged 
by the conversion ot nonrespondents to respondents. In this section we 
test the robustness of our error estimates by assuming that converting 
nonrespondents would have added cases like the extremes among the 
respondents. Specifically, we assume that the nonrespondents had the 
characteristics of the respondents at selected percentiles. In other 
words, for the Pth percentile we add to our respondents a number of cases 
equal to the number of nonrespondents all having the same amount of error 
as the individual respondent whose error value is such that P percent of 
all respondents had lower values. Table 7-9 summarizes the results of 
various assumptions about nonrespondent error. 
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VABLE 7-9. MMW ERROR NEASURES UNDER SEVERAL 
ASSUNPTIONS ABOUT NONRESPONDENT ERROR 



OVERALL PELL £ BBflB ^^^^^^^^^^ 

AVERAGE STH PERCENT ILE 10TN PERCEWTILE 9QTH PERCENTILE 95TH PERCENTILE ■■ TH E HEAN 



Observed Statistic 141.76 -416.50 

Total EstiMted 
Error ($H) 406 189 



->203*00 
272 



700*00 



624 



1050.00 



761 



70th 



Total Estimated 
Error ($M) 



mmi CAHP»S-6ASED NEED ERR OR ^^^^^^^^^^ 
AVERAGE STH PERCENTILE lOTH PERCENTILE 9Q T H PE RCENTILE 95TH PERCENTILE THE HEAN — 



Observed Statistic 373*10 



504 



-1173.45 



120 



-*700.00 
237 



1822.30 



863 



2968.75 



1153 



69th 



Observed Statistic 

Total Estimated 
Error ($M) 



nVFRALL CAHPUS-BAS ED AWARDS IN EXCESS OF NEED 

AVERAGE 7STH PERCENTILE 9QTH PERCENTILE 95TH PERCENTILE 
194.56 0 653.70 1146.85 



266 



217 



381 



505 



PERCENTILE OF 
THE MEAN 

82nd 



Observed Statistic 

Total Estimated 
Error ($M) 



nVFaAIL 6SL CERTIFICATION ERROR 
246.01 0 365*00 



9? 



681 



1036 



1032.79 
1686 



p:.rcentile of 

THE HEAN 

89th 
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For Pell error, the 5th, 10th, 90th and 95th percentiles were 
chosen. If all nonrespondents had error equal to the value observed at 
the 5th percentile (-$416.50), the estimate of error would fall to $189 
million. Using the error value at the 10th percentile, the estimate of 
error is $272 million. If all nonrespondents had error equal to the 
value observed at the 90th percentile ($700) the estima::e of error would 
increase to $624 million. Using the 95th percentile, the estimate of 
error is $761 million. Under the most extreme assumptions, the estimate 
of total Pell error could range from $189 million to $761 million. 

The same percentiles used for Peil error were used for Campus-Based 
need error. If all nonrespondents had error equal to the value observed 
at the 5th percentile, the estimate of error would fall to $120 million. 
Using the value observed at the 10th percentile, the estimate of total 
error is $237 million. If all nonrespondents had error equal to the 
value observed at the 90th percentile, the estimate error would increase 
to $863 million. Using the 95th percentile, the estimate of error is 
$1,153 million. Under the most extreme assumptions, the estimate of 
total Campus-Based need error could range from $120 million to $1,153 
million. 

For Campus-Based awards in excess of need, the 75th, 90th, and 95th 
percentiles were chosen. If all nonrespondents had error equal to the 
value observed at the 75th percentile (0) the estimate of error would 
fall to $217 million. Using the error value observed at the 90th 
percentile, the estimate of error would increase to $381 million. The 
estimate would be $505 million using the value observed at the 95th 
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I percentile. Under the most extreme assumptions, the estimate of total 

} Campus-Based awards in excess of need could range from S217 million to 

t 

I $505 million. 

\ 

^ The same percentiles that were used for the Campus-Based programs 

I were used for GSL. If all nonrespondents had error equal to the value 

observed at the 7bth percentile (0), the estimate of error would fall to 
$681 million. Using the value observed at the 90th percentile, the 
^ estimate of error would increase to $1,036 million. The estimate of 

error would be $1,686 million if the value observed at the ^5th 
percentile were used. Under the most extreme assumptions, the estimate 
of total GSL certification error could range from $681 million to $1,686 
million • 

7.2 VABIANCE ESTIMATES 

A critical part of the Findings Volume I Report are program-wide 
estimates of error in the population of Pell, Campus-Based, and GSL 
recipients. These estimates may vary from the actual population figures 
to the extent that the sample differs from the population. A series of 
45 tables in Appendix C presents selected estimates, standard errors of 
the estimates, and coefficients of variation. 

7.2.1 Variance Estimation by the Method of Balanced Repeated 
Replication 

The sampling error of an estimate, based on any sample design using 
any estimation procedure, no matter how complex, may be estimated by the 
method of replications. Theoretically, this method is equivalent to the 



idea that the sample selection/ collection of data, and estimation 
procedures are carried through independently (replicated) several times. 
In practice, random 50 percent subsets of the survey results are selected 
and estimates formed from each. The dispersion of the resulting 
estimates can be used to i.!*>asure the variance of the full sample. The 
method of replications has special advantages in reducing the complexity 
of variance computations. Another benefit is that it may be applied to 
compute sampling errors for higher-order statistics without the need for 
new variance expressions. 



The method consists of three steps: 

♦ Assemble data for the sample units that make up each of the 
replicates. This is equivalent to making a copy of the 
sample data for the units in each of the subsamples of the 
full sample. 

♦ Perform the estimation procedure on each of the replicates . 
The same estimation procedure, prepared for the full sample, 
is applied separately to each of the replicates. 

♦ Calculate the dispersion of the resulting estimates among the 
replicates to estimate the variance of the full sample. A 
relatively simple computation formula is used that does not 
depend on the form of the estimate for which the variance is 
to be approximated. 

Each of the half-sample replicates prepared for variance estimation 
must satisfy two criteria: 

♦ The replicate must comprise a sample approximately half the 
size of the full sample. 

♦ The selection of the half-sample must observe the same 
sampling principles ^>s the full sample. 



211 



For the replicates defined for the Title IV Quality Control Study 
design/ these criteria were satisfied by selecting half-samples of the 
units designated in the first stage of sampling in the full sample. For 
non-self-representing institutions/ a replicate comprised all students 
selected in half of the clusters; in self-representing institutions a 
replicate comprised half of the students selected at the school. 

Four of the eight certainty institutions were treated as individual 
strata. Half of the students in each institution were assigned to each 
half-sample. The remaining four certainty institutions were paired to 
form two additional strata. One of these was formed to reflect 
variability introduced by the sampling of branch campuses. The other was 
formed to create a sufficiently large stratum. Students in noncertainty 
institutions were assigned to a stratum and half -sample based on the 
geographic cluster of the institutions. Clusters were paired (in the 
order they appeared in the sampling frame) to form 50 strata. All 
students in a cluster were assigned to the same half -sample. 

Half-samples of the full sample were defined by randomly selecting 
one or the other half -sample from each of the 56 pairs; the number of 
different half-sample replicates possible by this method would equal 
2 (about 7.206 X 10 ). McCarthy has shown that the variance can 
be estimated with equivalent reliability from only a small number of 
orthogonal replicates.^ For this study^ the number of orthogonal 



^ Philip J. McCarthy/ Replication: An Approach to the Analysis of Data 
from Complex Surveys / National Center for Health Statistics/ Vital 
and Health Statistics Data Evaluation and Methods Research Series 2, 
No. 14 (Washington: Government Printing Office/ 1966). 
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replicates needed is the smallest multiple of four equal to or greater 
than the number of pairs ♦ With 56 pairs / the number of replicates needed 
is 56. 

To calculate the variance for estimate X calculated from the total 
sample, let be the estimate calculated from the ith half-sample, 
i = 1/ 2/ • . . / 56. The variance estimate for X is then 

56 

VAR (X) = (Xi - X)2 

56 rx=l 

This formula has been used to compute the standard errors (square root of 
the variance) for each statistic presented in Tables A-1 through 0-3 of 
Appendix C« 

7.2.2 Estimated Sampling Errors 

In Appendix C, we present estimated sampling errors developed using 
the methods described in the previous section. For each statistic^ we 
present the estimate itself/ the standard error of the estimate/ and the 
coefficient of variation (standard error of the estimate divided by the 
estimate) . 

The Title IV program and source of error are identified by letter as 
follows: 

A: Overall Pell Error 

B: Pell Student Error 

C: Pell Institutional Error 

D: Overall Campus^-Based Need Error 

E: Campus-Based Student Need Error 2^3 

F: Campus --Based Institutional Need Error 



G: Overall Campus-Based Payment Error (Awards in Excess of Need) 

H: Campus-Based Student Payment Error (Awards in Excess of Need) 

Is Cainpus-Sassd lastitutioaal Payment' Error (At/ards in Excess of 
Need) 

J: Overall Campus-Based Distributional Error 

K: Campus-Based Student Distributional Error 

L: Campus-Based Institutional Distributional Error 

M: Overall GSL Payment (Certification) Error 

N: GSL Student Payment (Certification) Error 

0: GSL Institutional Payment (Certification) Error 



The number associated with each table denotes the parameter for which 
population estimate, standard error and c efficient of variation are 
given: 



1: Populatio'i Total 

2: Number of Recipients with Error 

3: Mean Error per Recipient with Error 

*rhe statistics are given for all recipients as well as for groups of 
recipients defined by type and control of institution, dependency status, 
and type of aid received. 



7 •2.3 Revision of Estimated Total GSL Loan Volume 



After the GSL estimates and their standard errors were computed, an 
updated estimate of total GSL loan volume was made available by ED. 
Since a ratio adjustment was used to fix the estimated total population 
loan voliime based on our sample to the value of ED's estimate, some of 
the GSL figures must be adjusted accordingly. This revision will affect 
estimates of population totals and frequencies, as well as the standard 
errors associated with these. Each of these estimates should be 
decreased by about 10 percent. Estimates of means and percents are not 
affected nor are coefficients of variation for any estimate. 
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WESTAT 

An employ — -Gwnad Rmmmu-ct- Corooraoon 



February 6, 1986 



Ms. Kar«n Chauvin 

Offica oi Stud«nt Financial Aasistanca 

U.S. oapartaant of Education 

ROB #3, Room 5082 

7th and D Stra«ts, S.w. 

Washington, D. c. 20202 

Oaar Ms. Chauvin: 

A sampla composad of 300 institutions and 3,000 studants hns 
baan salactad for Staga II of tha Titl^ IV Quality Control Study. 
Lika many larga-scala studias, tha sanpla was intandad to satip v 
a numbar of spacific objactivas, and within cartain limitations 
it is particularly afficiant in masting thasa goals. Howavar, 
sinca tha sampla will ba a probability sampla of studants who 
participata in Titla IV programs, analysas do not hava to ba 
rastrictad to tha particular objactivas. Unbiasad astimatas can 
ba praparad for subdomains, and for cross-classification calls. 
Tha only constraint on tha analysas (and it is an important 
constraint) is that tha pracision is diractly ralatad to tha 
ntimbar of calls in tha cross-classification, and tha sampling 
arrors will incraasa with tha laval of datail. 

Tha pturposa of this lattar is to suggast within broad tanas, 
what wa baliava ara usaful ways of analysing tha data. Wa also 
indicata soma of tha faaturas of tha sampla and limitations that 
wara nacassary in achiaving tha major objactiva of tha pro j act. 

Tha actual siza of tha sampling arrors for this study will 
not ba known until tha projact has baan complatad and variancas 
conputad. Howavar, Wastat has praparad praliminary astimatas of 
sampling arrors for kay st?itistics, basad mainly on an analysis 
of tha pilot study (Staga 1) but also using othar sourcas. Thase 
astimatas can ba usad for ganaral planning purposas, but should 
ba raplacad by computad valuas whan survay data bacoma available. 
Tabla I sho%rs tha astiaatad coafficiantm of variation (CV) for 
tha total absoluta casa arrors for tha thraa programs. Tha CV's 
for othar- statistics can differ significantly froa these numbers. 
For example, tha CV»s fo? absolute casa error per student will be 
smaller than the value* sh«)wn, and they will ba even lower for 
absoluta case error per student with error. Conversely, they 
will be higher for subsets of students or institutions since the 
sample sizes will ba reduced. 
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we do not believe that the anticipated size of the sampling 
errors should preclude analysts from producing and analyzing 
JSdSain estimates. The limitations that exist are that 
analysts should take the sampling errors into account when 
dr'iing conclusions from the data. Essentially this means that 
the more detailed the cross classification, the greater the 
difference between two subdomains has to be before the analyst 
infers that a difference really exists. However, we can 
visualize the possibility that dramatic difference do exist 
between some subdomains, and this will be apparent even with 
large sampling errors. We should note that when the sampling 
errors are large, it may be good policy to call attention to the 
fact that an important difference exists but that there is some 
uncertainty about the actual size of the difference. 

In regard to the design of the sample, Westat carried out an 
evaluation of the efficiency of the planned design for Stage II, 
using estimates of the components of variance (prepared mostly 
from the pilot study) and a cost model utilizing earlier 
experience on related projects. The results indicated that the 
sample design — 300 institutions and 3,000 students — is close 
to an optimal design in meeting the principal objectives of the 
study. On the basis of component of variance analysis and the 
estimated overlap among programs, the total student sample for 
Stage II was allocated as shown in T^le 1. 

The term "optimal design" mentioned above, is not quite used 
in the conventional manner, and some comments may be helpful. 

o The efficiency of a sample partially depends on 
the amount' of information available that can be 
used for sample selection. Data used in 
establishing measures of size were much weaker for 
GSL than for the other two programs. This is a 
major reason for the expected CV on GSL being much 
higher than tor t" ^ other two programs. 

o It is sometimes possible to compensate for some 
features of the sample design in the estimation 
procedure, that is the weighting method, we 
expect to explore the possibility of reducing the 
sampling errors through ratio adjustment of the 
weights to known universe counts of institutions 
participating in each program. 

o In order to achieve the precision requirements, 
the sample design called for establishing a fixed 
overall sampling fraction for each program for the 
initially selected sample to achieve equal weights 
for all sample students in a program. As a 
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Ms. Kar«n Chauvin 
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February 6, 1986 



consttquttnctt# tha nunbar of students in th« sample 
will vary not only across programs within an 
institution but also among institutions for the 
same program. In about 20 institutions this 
variable worJdoad appeared to place a serious 
strain on tha field visits schedule and we 
subsaapled half of tha students in those 
institutions. As a result, the subsampled 
students will have double the weight, causing a 
departure from a self-weighting sample. 



Table 1. Stage II sample design for Title IV Quality Control 
Study 



Prograutt 


Direct 

S2UBple 

siza 


Sample 

counting 
overlap 


Adjusted 

for 
nonresponse 


Preliminary 
estimates 
of CV for total 
case errors 


Pell grants 


1,300 


2,482 


2,340 


6.16% 


C2uapus based 


1,511 


1,987 


1,755 


10.4% 


GSL 


400 


1,533 


1,331 


18.0% 



Wa would like to point out one additional feature of tha 
sample. In earlieif discussions of the sample design, there was 
considerable uncertainty about the degree of homogeneity within 
schools, and soma of the speculations were that intraclass 
correlations for estixBatas of total or mean errors might be as 
high as 0.5. The analysis of components of variance that we 
carried out indicates that tha intraclass correlations, although 
not trivial, are far below these levels, our current best 
estimates are that they average about 0.1. 

Sincerely, 

J^epn Waksberg 
Vice President 

JWs js 
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MEMORANDUM 



October 20, 1986 

Josefina A. Lago 

Sources of Variability in Overall Student Weights 
A. Introductica 

This memorandum documents the sources of variability in the 
final student weights and the variance implications of this 
variability. Two main sources are identified and discussed: 

Variability due to sample design features; 

Variability due to adjustments to the within-institution 
sampling rate in response to field results. 

As wiU be shown in A and B below, the variability introduced 
by the design features is quite small compared to l^at introduced by 
the adjustments to the v/ithin-institution sampling rates. 



To: 

From: 

Subject: 



B. Design Related Variability in Weights 

In the Title IV sample design Situdents were sampled 
independently for each of the three programs of interest but a 
student sampled for say PeU and receiving also Campus Based aid 
and GSL will also be included in the analysis for those other 
programs. This design feature resulted in a built-in variability m 
weights because the final weight of students receiving more than one 
type of Title IV aid must reflect the fact that they had multiple 
chances of coming into the sample. That is. since their probability ot 
selection is greater than it would have been if they had been listed m 
only one of the program frames, their final weight must be reduced 
accordingly. The effect of this built-in variabiUty in weights is 
evidenced by the variability in the median weights for the different 
nrooram combination. For students within Pell or CB the smallest 
meSan weight is 613 while the largest median weights are 1,815 ror 
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Pell and i,u7d lor v,d. rui w-*- tn^ -a- y - - 

The median weight of a GSL student who receives also FeU and CB is 
617 but for a GSL-only student it is 7,240. 

Another source of variability in weights that is common to 
practicaUy every survey is that which arises from nonresponse 
adjustment. For the Title IV survey, the institution nonresponse 
adjustinem factor (1.04) is quite small, but the stiident nonresponse 
adjustments will most likely be larger. 



C. Adjustments to Within-InstitatioB Sampling Rates 

The Title IV sample design caUs for developing within- 
institution sampling rates for the three programs of interest-Pell, 
Campus Based and GSL-such that the overall student SMnphng rate, 
and thus the overall student weight, would be the same for ^1 
students sampled for a particular program. The imoal overaU 
sampUng rates, fi , for the three programs ^ere: 

Pell: fi = 1,300/2,436,480 

CB: h = 1,511/1,358,014 

GSL: f3 = 400/3,247,000 

where the numerator is the target student sample size for the 
program, and the denominator is the best estimate available of the 
total number of students in the program. 

Next we describe the series of adjusunents applied to the ^ 
within-institution sampUng rates after the scheduling calls and 
throughout the field period and the variability introduced m the 
student final weights due to these adjusttnents. 

1) The schedule of visits to sample instimtions was developed 
based on the expected institution sample sizes ("takes") 
computed by applying the within-institution sampUng rates to 
the expected recipients reported on the Department of 
Education's universe files. After revised counts of recipients 
were obtained during the scheduUng calls to the sample 
institutions, Westat produced updated estimates of the 
expected institution "takes" based on the within-institution 
gQjjsjsj-jjt with a self-weighting sample. 



SHmuiUiS iaw9 
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Whenever the resulting case workioad couid not be 
aicommodated in the time aUocated to the institution, rates 
were cut to one half or one third of the original rate. At this 
point, rates were cut to one half or one tiurd of the ongmal rate 
for 33 institutions. 

2) At several institutions, when the dau coUector arrived s/he 
found that the actual number of recipients in one or more of 
Uie Titie IV programs was considerably larger Uian tiiat 
reported during the scheduling call and could not be 
accommodated^ during the time allotted to Uie visit. After 
calling Advanced Technology cases were subsampled (generally 
a one-half subsample) to fit the time available for the visit . 
This resulted in a smaller witiiin-institution sampUng rate than 
tiiat computed to yield a self-weighting sample. 

3) Half way through the field period our estimate of the total 
sample yield was only 2.600 students. The situation was 
discussed with the Department of Education and Uie decision 
was made to increase the "take" m any institution where Uie 
schedule permitted it in order to achieve a total sample of 
3,000 students. This procedure, particularly in cases where Uie 
"take" for Uie program was initially small, resulted in some 
significant weight reductions. For instance, if Uic initial "take" 
was one student and Uie data collector estimated Usat s/he 
could do five students, by increasing Uie sample to five Uie 
weight was reduced by a factor of five. 

4) The same student may have been sampled from more than 
one list because Uie student sampling was carried 
independentiy for each program. For twelve students Uieir 
joint probabUity of selection was cut in half to take into account 
Uie fact that only one questionnaire was obtained for them 
although Uiey came into Uie sample twice. 

The adjustment described in 1, 2 and 4 above had Uie effect of 
reducing Uie overall student selection probability and Uius 
increasing Uie weight. The adjustment described in 3 resulted in an 
increase in Uie overall probabiUty of selection and Uius decrease in 
Uie weight Qearly. as shown in Table 1. for recipients of a given 
program Uie variabiUty in weights accounted for by Uiese sources is 
about ten to twenty times larger Uian Uiat accounted for by me 
built-in variability discussed in B. 



Table 1. Oitlribullon ol Title IV Weights Before Truncation and Effect of Truncarkm on Estimalee of ReciplenH 



- 

Sum of 
weights 


Program 


lyledlM 


Sample size 


Top 88 
percent 


Top 86 
percent 


Bottom 1 
percent 


Weighlsd 
recipients 


Expected* 


161 


1 CBonly 


1075 


148 


4838 


1667 


« 
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Before Trttnoatlc 
Pel 2244000 


a: 

2436480 


303 


2 CB/GSL 


d36 


317 


2620 


1658 


61 


CB 1363000 


1358014 


1626 


3 QSLoniy 


7240 


248 


14728 


7240 


574 


Q6L 2846000 


3247000 


756 
471 
428 
569 


4 Pel only 

5 PeNA^B 

6 PeH/CB/QSL 

7 PMGSl 


1815 
675 
618 

1451 


415 
718 
718 
418 


6741 
2074 
1704 
4683 


3630 
1351 
875 
2418 


86 
58 

102 
480 


After Truncation 
Pen 2177000 

CB 1312000 

j QSL 2868000 


2436480 
13580U 
3247000 



*Ba&ed on Stage I 



\22 




D. Impact of the Variability in Weights on Variances for 
Program Estimates 

From the point of view of precision of sample estimates for a 
particular program, a self-weighting sample (one where every 
student in a program has the same probability of selection) is 
preferable to one where the smdents have varying probabilities 
within a program. As shown m Table 1, the Title IV weights for a 
particular program, for the reasons discussed above, have 
considerable variability. Large extreme weights are generally of 
greater concern that low extreme weights because they can result m 
significant increases in the variances of survey estimates. The 
undesirable consequence of cases with small extreme weights is that 
they make a very small <:ontribution to the total estimate as if the 
sample size were smaUer that what it actually is. Thus, after 
evaluating the results shown in Table 1 it was decided that the top 5 
percent of the large weights would be truncated to the 95-percetttile 
value shown in the table, and the bottom 1 percent of the low 
weights would be truncated to the 1 -percentile value. 

After trunca^on, the weights are distriw^ued as shown below. 

Median Lowest Highest V2 

Pell: 675 • 59 3630 .3828 

CB: 675 59 1667 .1579 

GSL 936 61 7240 1.5964 

The impact on variances of this variability in weights may be 
approximated by the factor 

F= 1 + V2 

where V2 is the relvariance (square of the coefficient of variation) of 
the weights. Thus, because of the variabUity in weights, the vanance 
of PeU estimates will be increased by a factor of 1.38, for CB by 1.16 
and for GSL by 2.60. Relating these results to the preUminary 
estimates of CVs for total case error shown in Joe Waksberg's 
memorandum (February 6, im\ the CVs with and without the 
effect of the variability in weights are presented below. 



Pfcliuiiu&ry 
estimates of CVs 
Program for . total case error 

PeU . 6.15 % 
CB 10.4 % 

GSL 18.0 % 



Impact of 
variability 
in weights 

1.18 
1.08 
1.61 



Adjusted 
CVs 

7.3% 

11.2% 

?.8,9% 



cc: J. Waksberg 
R. Learmonth 



Tabto 1. Dislnbulion of Titto IV Woighls Botoro Trurcalkm and Effect of Tmncaioo on EtIimalM of ftectptooii 



•IP 



Sinn of 


Program 


1 

Modum 


Sampto fti20 


lop w 
parcont 


paroant 


Duimn 1 
parcant 


radptoata 


Expaciad* 
















Satora Truacalia 


a: 




161 


1 CBonly 


lU/O 


140 




109/ 




M 


2244000 




2436480 


303 


2 CB/GSL 


036 


317 


2620 


1658 


61 


CB 


1363000 




1358014 


1626 


3 GSLonly 


7240 


248 


14720 


7240 


574 


GSL 


2946000 




3247000 


V56 


4 Pol only 


U)15 


415 


6741 


3630 


06 


Aflar 
Pad 


Truaoatloi 

2177000 


: 


2436480 


471 


6 PaH/CB 


675 


710 


2074 


1351 


50 


CB 


1312000 




1358014 


428 


6 PeM/Ce/6SL 


618 


718 


1704 


875 


102 


G8L 


2869000 




3247000 


589 


7 Peil/QSL 


1451 


416 


4683 


2418 


490 











*Basod on S^go t 
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TnXE IV QUALITY CONTROL PROJECT 

STAGE n: AN INTEGRATED PROaECT 



Background 

The Title IV Student Financial Assistance programs have grown dramatically in both 
dollar volume and student participation during the past decade. With this rapid growth 
has come the potential for errors in student application information, student eligibility 
certification, award calculations, and other program procedures. The Department of 
Education is increasingly aware of the need to reduce these errors and to improve 
performance in aU Federal student assistance programs. The Department is committed to 
ensuring that these programs operate efficiently and that funds are allocated properly. 

Project Objectives and Activities 

The two stages of the Title IV QuaUty Control Project are designed to measure the 
degree to which errors exist In the delivery of aU five major sources of student aid, 
identify causes of error, recommend management corrective actions, and provide techni- 
cal assistance and follow-up analysis. Stage I was a pilot study to determine the 
feasibility of measuring error in the Campus-Based programs and the Guaranteed Student 
Loan (GSL) certification process. Included in the pUot were the National Direct Student 
Loan (NDSL), Supplemental Educational Opportunity Grant (SEOG), and College Work- 
Study (CWS) programs, known as the "Campus-Based" programs, and the Guaranteed 
Student Loan program. In Stage n the scope has been expanded to include all five major 
Title IV student assistance programs by adding the Pell Grant program. 

The primary activities conducted during Stage I (1984) included the following: 

• The 1983-8* program-wide error rates by number of cases and amount of 
dollars for the Campus-Based and GSL programs were documented, computed, 
and analyzed. 

• Institutional compliance with Federal legislation and regulations, school- 
specific packaging philosophies, need analysis principles, and other school 
administrative policies and procedures were documented and analyzed. 

• The major types of program errors were Identified and analyzed. 

• The effectiveness of quality control procedures and corrective actions which 
had already been implemented to reduce or prevent error was evaluated. 

• Recommendations were developed for management actions to correct each of 
the major errors identified. 

These activities were based on data gathered through a nationwide survey of 820 
students and their parents, and 281 postsecondary institutions in the spring of 
Trained interviewers visited a random, representative sample of pubUc, private, and 
proprietary institutions. At each Institution, the Interviewer selected a random sample of 
Campus-Based and GSL recipients and reviewed their financial aid records. The financial 
aid administrator was also Interviewed and asked to desalbe the Institution's student aid 



awarding preeedures. Another group oi experiefHred interviewers visited the students who 
were selected, and their pwents, and asked them to supply documents verifying the 
Jnlormation that appeared on their application forms. 

Findings from Stage I suggest that error can be quantified and Is significant fn the 
Campus-Based program and GSL certification process. Based on these findings, a broader 
effort wlU be undertaken In Stage H to more precisely d^lr|e= and^measure^^ and 
obtain a more comprehensive understanding of PeU, CaiiipuffiBased, and GSL P«'0P'«'n 
error. In Stage n, the 1985-S6 program-wide error rate for the five programs wUl be 
measured, pr^able causes for malor types of error wlU'be Identified, and recommenda- 
tions for corrective actions wlU be developed. In particular, tt.4 Pell error rate will be re- 
measured to determine the effectlven^ of recently expanded validation requirements. 
Stage n will also evaluate the effectiveness of other institutional quality control 
procedures in reducing payment error. 

Stage n methodology will be the same as In Stage It a representative sample (about 
300) Institutions will be visited; student record information collected; financial aid 
administrators Interviewed; and students (about 3,000) and thslr parents asked to provide 
documents verifying their application Information. 

Project Contractor 

The Office of Student Financial Assistance awarded the Title IV Quality Control 
Project contract in January 19W to Advanced Technology, Inc., of Reston, Virginia, and 
its subcontractor, Westat, Inc., of RockvUle, Maryland. Interviewers who visit institutions 
are employees of Advanced Technology; Interviewers who visit students and their parents 
are employees of Westat. Ms. Carol ML^ler of Advanced Technology is ths. project 
director. Mr. Robert Learmonth Is the project team leader for Westat. 

Sponsoring Agency 

The Title IV Quality Control Project is sponsored by the Statbtical Analysis Branch 
of the Division of Quality Assurance, Debt Collection and Management Assistance 
Service, VJS, Department of Education. Dr. David Iwamoto is the chief of the Statistical 
Analysb Branch, and Mr. Ernst Becker is the director of the Division of QuaUty 
Assurance. 

Additional information may be obtained from Ms. Karen Chauvin, project officer for 
the Statistical Analysis Branch, Division of QuaUty Assurance at (202) 2*5-0102. 
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Tsblc A- 1. Ei^t'*"''^ sampling etrots (op total Pell error 
Overall error, $50 tolerance 



Category 


Estimate 
(SmiUions) 


Standard error 
(Smiilions) 


Coefficient 
of variation 


All recioients 




$43.58 


0.107 


of institution 








nublic 
4-ycar public 

4-ycar private 
Proprictaxy 


$61.89 
$178.67 
$18.42 
$85.63 
$61.84 


$24.84 
$21.14 
$15.79 
$21.19 
$28.95 


0.401 
0.118 
0.857 
0.247 
0.468 


Dependency status 








Independent 
Dependent 


$301.24 


?3n 44 
$24.72 


0.289 
0.082 


Type of aid received 








Pell only 
Pell and C-B 
Pell and GSL 
PeU, C-B and GSL 


$138.42 
$76.62 

$109.00 
$82.41 


$32.49 
$11.46 
$25.08 
$10.68 


0.235 
0.150 
0.230 
0.130 
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Table A-2. Esdassttd sampling errw-rs for numbe" of recipients with Pell ciron 
Overall error, $50 tolerance 



Category 


Estimate 
(thousands) 


Standard error 
(thousands) 


Coefficient 
of variation 




1520.76 


55.31 


0.036 


Type and control 
of institution 








4-year public 
2-year private 
4-year private 
Pxtjprietary 


468.20 
564.87 
44 59 
256.38 
186.72 


53.58 
50.50 
22.12 
37.09 
37.95 


0.114 
0.089 
0.496 
0.145 
0.203 


Dependency status 








Independent 
Dependent 


CA'i QO 

542.W 
977.77 


38.94 


0 070 
0.040 


Type of aid received 








Pell only 
Pell and C-B 
Pell and GSL 
Pell, C-B and GSL 


546.45 
312.66 
384.21 
277.44 

■ — 


46.83 
25.34 
35.13 
21.91 


0.086 
0.081 
0.091 
0.079 
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Table A-3. Estimated sampling enors for mean error per recipient with Pell error: 
Overall error, $50 tolerance 



Category 


Estimate 


Standard error 


Coefficient 
of variation 


All recipients 


$267.27 


$29.42 


0.110 


Type and control 
of institution 








2-ycar public 
4-ycar public 

4-ycar private 
Proprietary 


$132.18 
$316.31 
$413.22 
$333.98 
$331.19 


$55.41 
$29.74 

$305.90 
$56.11 

$147.28 


0.419 
0.094 
0.740 
0.168 
0.445 


Dependency status 








Independent 
E)ependent . 


$193 75 
$308.09 


$56.22 
$24.58 


0.290 
0.080 


Type of aid re( jived 








Pell only 
PeUandC-B 
Pell and GSL 
PeU, C-B and GSL 


$253.3^ 
$245.C 
$283.70 
$297.03 


$54.18 
$36.44 
$54.10 
$37.06 


0.214 
0.149 
0.191 
0.125 

1 
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Table B-1. Esdmated sampling errors total Pell error 
Student error, $50 tolerance 



Category 


Estimate 
($millions) 


Standard error 
(Smillions) 


Coefficient 
of variation 


AH recipients 


$272.17 


$24.33 


0.089 


of institution 








2-ycar public 
4-ycar public 

4-ycar private 
Proprietary 


$40.12 
$125.07 
$7.94 
$50.85 
$48.17 


$14.20 
$17.16 
$3.31 
$8.37 
$17.82 


0.354 
0.137 
0.417 
0.165 
0.370 


Dependency siaius 








Independent 
Dependent 


$241.35 


$13 01 
$18.85 


0.422 
0.078 


Type of aid received 








Pell only 
Pell and C-B 
Pell and GSL 
Pell, C-B and GSL 


$92.53 
$55.16 
$64.27 
$60.20 


$18.69 
$8.7: 

$13.87 
$9.52 


0.202 
0.158 
0.216 
0158 
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I'abie o^z* lismnaica sampunij canui^ ?.ui uumuvi vm. twn/iwuw • 

Student cirar, $50 tolerance 



Category 


Estimate 
(thousands) 


Standard error 
(thousands) 


Coefficient 
of variation 


AH recipients 


904.74 


44.27 


0.049 


Type and control 
of institution 








2-year public 
4-ycar public 
2-ycar private 
4-year private 
Proprietary 


190.53 
371.87 
25.94 
216.19 
100.22 


26.71 
37.42 
11.15 
30.64 
25.12 


0.140 

0.430 
0.142 
0.251 


Dependency status 








Dqwndent 


185.41 
719.33 


22.04 
34.51 


0.048 


Type of aid received 








Pell only 
PcUandC-B 
Pell and GSL 
Pell, C-B and GSL 


285.76 
187.47 
222.71 
208.80 


35.24 
18.93 
22.24 
19.59 


0.123 
0.101 
0.100 
0.094 
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Table B-3. Esdniatcd sampling emws for niean ciror per reclpiem with Fell cffof : 
Student error, $50 colerance 



Category 


Estimate 


Standard error 


Coefricient 
of variation 


All recioitnts 


$300.82 


$20.01 


0.067 


of institution 








4-ycar public 
2-year pnvaic 
4-year private 
Proprietary 


$210.60 
$336.34 
$306 30 
$235.23 
$480.66 


$72.46 
$33.77 

$148.65 
$33.50 

$133.24 


0.344 
0.100 
0.485 
0.142 
0.277 


Dependency status 








Independent 
Dependent 


$335.52 


$22.19 


0.365 
0.066 


Type of aid received 








Pell only 
Pell and C-B 
PeUandGSL 

Pell, C-BandGSL 


$323.79 
$294.26 
$288.59 
$288.33 


$47.51 
$46.18 
$52.14 
$40.27 


0.147 
0,157 
0.181 
0.140 



Table C-1. EsdiBated sampling enors far total Fcil error 
bistitutional error, $50 tolerance 



Category 


Estimate 
(SmilUons) 


Standard error 
(Smillions) 


Coefficient 
of variation 


All recipients 


$133.36 


$42.85 


0.321 


of institution 








2- vear nublic 
4-year public 
z-ycoT pnv cue 
4-ycar private 
Proprietary 


$21.55 
$53.53 
$10.45 
$34.26 
$13.57 


$20.05 
$14.97 
$14.52 
$16.44 
$22.29 


0.931 
0.280 
1.389 
0.480 
1.643 


Dependency status 








Independent 
Dependent 


$59.16 


$32 16 
$17.23 


0.433 
0.291 


Type of aid received 








Pell only 

n^ll ^^A r>_D 

PeUandGSL 
Pell, C-B and GSL 


$46.00 
$21.05 

$22.00 


$26.17 
$9.38 

$20.37 
$5.58 


0.569 
0.446 
0.460 
0.254 



Tabic Estixriatcd sampling citots for number of recipients with Pell CxxkjT. 
Institutional error, $50 tolerance 



Category 


Estimate 
(thousands) 


Standard error 
(thousands) 


Coefficient 
of variation 


AH recipients 


839.18 


59.36 


0.071 


Tvn^ and control 
of institution 








2-year public 
4-year public 
^•vear orivate 
4-year private 
Proprietary 


351.97 
257.01 
32.13 
74.50 
123.57 


46.05 
37.38 
18.83 
18.53 
28.51 


0.131 
0.145 
0.586 
0.249 
0.231 


Dependency status 








indcpenueni 
Dependent 


436 85 
402.33 


34.74 
40.17 


0.080 
0.100 


Type of aid received 








Pell only 
PcUandC-B 
PeUandGSL 
Pell. C-B and GSL 


334.23 
174.89 
228.04 
102.03 


37.74 
21.58 
29.29 
13.58 


0.113 
0.123 
0.129 
0.133 

. _ 
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Table C-3. Estimated sampling errors for mean error per recipient with Pell error: 
Institutional error, $50 tolerance 



Category 


Estimate 


Standard error 


Coefficient 
of variation 


All recipients 


$158.91 


$51.69 


0.325 


Type and control 
of institution 








2-ycar public 
4-year public 
2-vcarDrivatc 
4-year private 
Proprietary 


$61.22 
$208.28 
$325.37 
$459.85 
$109.78 


$59.40 
$44.08 
$427.20 
$149.90 
$212.35 


0.970 
0.212 
1.313 
0.326 
1.934 


Dependency status 








Independent 
Dependent 


$169 84 
$147.05 


$75.26 
$40.80 


0.443 
0.278 


Type of aid received 








Pell only 
FeUandC-B 
PeUandGSL 
PeU, C-B and GSL 


$137.62 
$120.37 
$194.30 
$215.65 


$76.30 
$51.76 
$83.95 
$60.04 


0.554 
0.430 
0.432 
0.278 



r lablc Li*i. r!.Si miffs'?* ^oiiipun5 va*\/*<» 104 %^%*** ^ *^ ^ — . 

I Overall error, $50 tolerance 

I 



Category 


Estimate 
(Smillions) 


Standard error 
(Smillions) 


Coefflcient 
of variation 


AW recipients 


$504.61 


$58.15 


0.115 


Tvne and control 
0/ institution 








2-ycar public 
4-year public 

4-ycar private 
Proprietary 


$55.44 
$207.64 
$5.40 
$214.54 

$21.59 


$26.98 
$37.78 
$8.08 
$42.39 
$19.22 


0.487 
0.182 
1.496 
0.198 
0.891 


Dependency status 








Independent 
Dependent 


$262.75 


$43.74 
$41.06 


0.181 
0.156 


Type of aid received 








C-BOnly 
Pell and C-B 
C-B and GSL 
Pell, C-B and GSL 


$81.62 
$106.37 
$216.42 
$100.21 


$15.70 
$32.48 
$38.10 
$30.56 


0.192 
0.305 
0.176 
0.305 
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Tabic D-2. Estiinaicd sampling wfOfs for number of recipients with C-B used ctrt: 
Overall error, $50 tclcrance 



Category 


Estimate 
(thousands) 


Standard error 
(thousands) 


Coefficient 
of variation 


All recipients 


990.42 


29.59 


0.030 


Tvne and control 
of institution 








2-Year public 
4-year public 
^-vftflT tyrivate 
4-ycar private 
Proprietary 


169.67 
417.55 

21.34 
341.70 

40.15 


28.64 
42.11 
9.66 
35.47 
11.84 


0.169 
0.101 
0.433 
0.104 
0.295 


Dependency status 








Independent 
Dependent 


659.90 


91 71 

23.52 


0 066 
0.036 


Type 0/ oirf received 








C-B Only 
PeUandC-B 
C-BandGSL 
Pell, C-B and GSL 


123.27 
336.11 
247.23 
283.81 


12.64 
25.80 
23.18 
20.57 


0.103 
0.077 
0.094 
0.073 



ERIC 
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-j-guig £5.3^ cgjiaiatsd sampling errors for mean error per recipient with C-B need error 
Overall error, $50 tolerance 



Category 


Estimate 


Standard error 


Coefficient 
of variation 


All recioients 


$509.50 


$62.81 


0.123 


I yp€ una cumrui 

of institution 








?«vftair nuhlic 

4-ycar public 
/-year pnvaie 
4-year private 
Proprietary 


$326.78 
$497.28 
$253 20 
$627.85 
$537.64 


$164.18 
$95.03 
$443.92 
$102.77 
$584.13 


0.502 
0.191 
1.753 
0.164 
1.087 


Dependency status 








Independent 
Dependent 


C711 TQ 
•9/01. 10 

$398.16 


$62.79 


0 158 
0.158 


Type of aid received 








C-B Only 
Pell and C-B 
C-B and GSL 
PeU, C-B and GSL 


$662.14 
$316.46 
$875.39 
$353.08 


$122.02 
$100.96 
$115.14 
$110.62 


0.184 
0.319 
0.132 
0.313 



lERJC 
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Table E-1. Esrimatcd sanraling errors for total C-B need error 
Student error, $50 tolerance 



Category 


Estimate 
(SmHlions) 


Standard error 
(Smiilions) 


Coefficient 
of variation 


A 11 r^cioi^nts 


$403.56 


$51.42 


0.127 


Typ€ dM COttlrOi 

of institution 








x*yCiU puuuw 

4-ycar public 
2-ycar private 
4-ycar private 
Proprietary 


$56.23 
$175.08 

IS? fl'^ 

$153.22 
$17.01 


$19.15 
$33.40 
$11.30 
$35.42 
$13.97 


0.341 
0.191 
5.573 
0.231 
0.822 


Dependenc:^ status 








Independent 
Dependent 


$221.53 
$182.03 


on 
$36.17 


0 207 
0.199 


Type of aid received 








C-B Only 
Pell and C-B 
C-B and GSL 
Pell, C-B and GSL 


$46.69 
$117.34 
$148.35 

$91.17 


$12.68 
$28.25 
$30.28 
$24.70 


0.272 
0.241 
0.204 
0.271 
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Table E-2. Estimated sampUc > errors for number of recipients with C-B need error: 
Student error, $50 tolerance 



Category 


Estimate 
(thousands) 


Standard error 
(thousands) 


Coefflcient 
of variation 


All recipients 


826.70 


26.12 


0.032 


TvM and control 

of institution 








2-ycar public 
4-ycar public 

4-ycar private 
Proprietary 


130.10 
343.74 

19.24 
301.07 

32.55 


21.38 
34.74 

8.92 
32.80 

9.24 


0.164 
0.101 
0.464 
0.109 
0.284 


Dependency status 








Independent 
Dependent 


274 58 
552.12 


19.95 
23.58 


0.073 
0.043 


Type of aid received 








C-B Only 
Pell and C-B 
C-B and GSL 
PeU, C-B and GSL 


107.21 
269.94 
220.65 
228.89 


12.49 
21.95 
21.14 
16.12 


0.117 
0.081 
0.096 
0.070 
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Table Estimated sampling errars for mean error per recipient with C-B need error 
Student csfoi', $50 coicfEnce 



Category 


Estimate 


Standard error 


Coefficient 
of variation 


AH recipients 


$488.15 


$63.42 


0.130 


Tvn* and control 
of institution 








2-ycar public 
4-ycar public 

4-year private 
Proprietary 


$432.18 
$509.32 
$105.38 
$508.9? 
$522,39 


$154.98 
$99.12 
$633,65 
$101.05 
$493.66 


0.359 
0.195 
6.013 
0.199 
0.945 


Dependency status 








Independent 
Dependent 


<:q06 80 
$329.68 


$139.61 
$62.81 


0.173 
0.191 


Type of aid received 








C-B Only 
PeUandC-B 
C-BandGSL 
PeU. C-BandGSL 


$435.52 
$434.68 
$672.33 
$398.32 


$110.08 
$106.91 
$115.27 
$107.70 


0.253 
0.246 
0.171 
0.270 
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Table F- 1 . Estimated sampling eirors for toad C-B need error 
Institutional cnor, $50 tolerance 



Category 


Estimate 
($inillioRs) 


Standard error 
(Smillions) 


Coefficient 
of variation 


All recipients 


$100.64 


$34.94 


0.347 


Type and control 
of institution 








2-ycar public 
4-year public 
2-year private 
4-year private 
Proprietary 


-$0.86 
$32.35 

$3.41 
$61.06 

$4.68 


$13.08 
$13.82 

$7.01 
$26.30 

$9.50 


15.219 
0.427 
2.053 
0.431 
2.031 


Dependency status 








inocpcnuci i t 
Dependent 


$20.32 
$80.32 


$19.64 
$26.81 


0.966 
0.334 


Type of aid received 








C-B Only 
Pell and C-B 
C-BandGSL 
Pell, C-B and GSL 


$35.01 
-$10.98 
$67.56 
$9.05 


$11.39 
$15.78 
$24.56 
$17.62 


0.325 
1.437 
0.364 
1.947 
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Table F-2. Estimated sampling errors for number of recipients with C-B need error 
Institutional error, $50 tolerance 



Category 


Estimate 
(thousands) 


Standard error 
(thousands) 


Coefficient 
of variation 


AH recipients 


403.50 


23.57 


0.058 


Type and control 
of institution 








2-ycar public 
4*ytar public 
2-vear Dxivate 
4-year private 
Proprietary 


96.01 
168.50 
9.13 
103.50 

26.36 


19.59 
19.90 
4.70 
15.24 
10.00 


0.204 
0.118 
0.515 
0.147 
0.379 


Dtpendency status 








Independent 
Dependent 


1 J J. JO 

250.12 


13 53 
18.38 


0.O88 
0.O74 


Typ4 of aid received 








C-B Only 
PeUandC-B 
C-B and GSL 
PeU, C-B and GSL 


34.14 
180.29 

59.01 
130.05 


7.55 
15.66 

9.91 
13.44 


0.221 
0.O87 
0.168 
0.103 
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Tabic F-3. Estimated sampling cirors for nwan error per recipient with C-B need crron 
Institutional error, $50 tolerance 



Category 


Estimate 1 


Standard error 


CoefHcient 
of variation 


All recipients 


$249.43 


$90.88 


0.364 


Type and control 
of institution 








2-year public 
4-year public 
2-year private 
4-year private 
Proprietary 


-$8.95 
$191.98 
$373.89 
$589.99 
$177.42 


$137.21 
$89.35 . 
$924.03 
$243.83 
$411.82 


15.326 
0.465 
2.471 
0.413 
2.321 


Dependency status 








inacpcuuciii 
Dependent 


$132 50 
$321.13 


$130.77 
$110.69 


0.987 
0.345 


Type of aid received 








C-B Only 
Pell and C-B 
C-B and GSL 
Pell, C-B and GSL 


$1,025.56 
-$60.90 

$1,144.81 
$69.60 


$278.17 
$84.91 
$313.07 
$138.28 


0.271 
1.394 
0.274 
1.987 
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Tabic G-1. Estimated sampling errors for total C-B paynwnt error 
Overaii error, $50 toicrancc 



Category 


Estimate 
(Smillions) 


Standard error 
($miliions) 


Coefficient 
of variation 


All recipients 


$266.27 


$21.25 


0.080 


Type and control 
of institution 








2-ycar public 
4-ycar public 
2-ycar private 
4-ycar private 
Proprietary 


$36.25 
$yy. / z 
$4.29 
$115.22 
$10.79 


$10.47 

$2.79 
$i8.64 
$6.83 


0.289 
0 157 
0.651 

U.iOZ 

0.633 


Dependency status 








Tn HeneTidcnt 

Dependent 


$101.49 
$164.78 


$15.38 
$19.25 


0.152 
0.117 


Type of aid received 








C-B Only 
Pell and C-B 
C-B and GSL 
PeU, C-B and GSL 


$43.41 
$82.62 
$83.64 
$56.59 


$8.12 
$14.56 
$15.23 

$8.11 


0.187 
0.176 
0.182 
0.143 
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Table G-2. Estimated sampling eirors for number of recipients with C-B payment error: 
Overall error, $50 tolerance 



Category 


Estimate 
(thousands) 


Standard error 
(thousands) 


Coefficient 
of variation 


All recipients 


289.06 


16.47 


0.057 


Type and control 
of institution 








2-yccr public 
4-ycar public 
2-vcar Dtivatc 
4-year private 
Proprietary 


45.50 
132.23 
2.61 
97.98 
10.73 


9.75 
14.98 

1.55 
14.04 

4.76 


0.214 
0.113 
0.593 
0.143 
0.443 


Dependency status 








Independent 
Dependent 


179.16 


10.80 
15.48 


0.098 
0.086 


Type of aid received 








C-B Only 
Pell and C-B 
C-BandGSL 
PeU, C-B and GSL 


47.75 
98.29 
79.86 
63.16 


8.02 
11.45 
12.20 

8.68 


0.168 
0.117 
0.153 
0.137 
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Table G-3. Estimated sampling eirors for mean error per recipient with C-B payment error 
Overall error, $50 tolerance 



Category 


Estimate 


Standard error 


Coeffldent 
of variation 


All recipients 


$921.14 


$61.71 


0.067 


of institution 








2-ycar public 
4-year public 

^'YCoT pnvoic 

4-year private 
Proprietary 


$796.72 
$754.17 
$1,641.42 
$1,175.87 
$1,004.83 


$167.54 
$73.92 
$451.95 
$105.76 
$503.43 


0.210 
0.098 
0.275 
0.090 
0.501 


Dependency status 








Independent 
Dependent 


$Q13 37 
$919.77 


$90.36 
$73.44 


0.098 
0.080 


Type of aid received 








C-B Only 
PeUandC-B 
C-B and GSL 
PeU,C-3andGSL 


$909.15 
$840.61 
$1,047.23 
$896.08 


$99.73 
$113.40 
$105.38 
$119.09 


0.110 
0.135 
0.101 
0.133 
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Table H-1. Estimated sampling errors for total C-B payment error 
Student error, $50 tolerance 



Category 


Estimate 
(Smillions) 


Standard error 
(Smillions) 


Coefficient 
of variation 


All recipients 


$137.74 


$15.28 


0.111 


Type and control 
of institution 








2-ycar public 
4-ycar public 
2-ycar private 
4-ycar private 
Proprietary 


$21.86 
$62.51 

$2.75 
$47.14 

$3.49 


$7.61 
$12.49 

$2.43 
$11.73 

$3.29 


0.348 
0.200 
0.884 
0.249 
0.943 


Dependency status 








Dependent 


$57.10 
$80.64 


$9.49 
$14.71 


0.166 
0.182 


Type of aid received 








C-B Only 
PeU and C-B 
C-B and GSL 
PcU, C-B and GSL 


$20.54 
$53.44 
$43.96 
$19.81 


$5.76 
$11.95 
$10.72 

$4.84 


0.280 
0.224 
0.244 
0.244 
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Table H-2. Btimated sampling errors for number of rscipients with C-B payment error: 
Student error, $50 tolerance 



Category 


Estimate 
(thousands) 


Standard error 
(thousands) 


Coefficient 
of variation 


All recipients 


163.12 


13.21 


0.081 


Type and control 
of institution 








2-ycar public 
4-ycar public 
2-ycar private 
4-ycar private 
Proprietary 


26.68 
81.60 

1.65 
50.07 

3.12 


6.07 
11.15 

1.29 
10.74 

2.24 


0.228 
0.137 
0.778 
0.214 
0.719 


Dependency status 








JJlUvpwIiUCilw 

Dependent 


64.57 
98.55 


7.66 
12.53 


0.119 
0.127 


Type of aid received 








C-B Only 
Pell and C-B 
C-B and GSL 
Pell, C-B and GSL 


26.48 
60.98 
48.99 
26.67 


6.20 
9.15 
10.47 
5.13 


0.234 
0.150 
0.214 
0.192 
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Table H-3. Estimated sampling errors for mean error per recipient with C-B payment error 
Student error, $50 wierance 



Category 


Estimate 


Standard error 


Coefficient 
of variation 


All recipients 


$844.44 


$66.04 


0.078 


Type and control 
of institution 








2-ycar public 
4-ycar public 
2-ycar private 
4-ycar private 
Proprietary * 


$819.28 
$766.07 

$1,665.52 
$941.51 

$1,117.04 


$213.00 
$90.94 
$637.66 
$112.56 
$577.12 


0.260 

0.383 
0.120 
0.517 


Dependency status 








Independent 
Dependent 


$884.32 
$818.31 


$104.34 
$89.00 


0.118 
0.109 


Type of aid received 








C-B Only 
Pell and C-B 
C-B and GSL 
Pell, C-B and GSL 


$775.83 
$876.26 
$897.24 
$742.80 


$128.39 
$152.24 
$91.29 
$121.84 


0.166 
0.174 
0.102 
0.164 
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Table I-l. Estimated sampling errors for total C-B payment error 
Institutional error, $50 tolerance 



Category 


Estimate 
(Snillions) 


Standard error 
(Smillions) 


Coefficient 
of variation 


All recipients 


$136.26 


$15.17 


0.111 


Type and control 
of institution 








2-ycar public 
4-ycar public 
2-ycar private 
4-ycar private 
Proprietary 


$15.35 
$40.10 

$2.10 
$71.13 

$7.58 


$4.82 
$6.50 
$1.59 
$13.50 
$4.17 


0.314 

0.754 
0.190 
0.550 


Dependency status 








lllUCpvUUvIlt 

Dependent 


$47.84 
$88.42. 


$9.94 
$13.25 


0.208 
0.150 


Type of aid received 








C-B Only 
PeUandC-B 
C-B and GSL 
PeU.C-BandGSL 


$24.62 
$30.94 
$41.42 
$39.28 


$5.74 
$6.32 
$12.44 
$6.93 


0.233 
0.204 
0.300 
0.176 
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Table 1-2. Estimated sampling errors for number of recipients with C-B payment crron 
Institutional error, $50 tolerance 



Category 


Estimate 
(thousands) 


Standard error 
(thousands) 


Coefflcient 
of variation 


All recipients 


171.89 


14.06 


0.082 • 


Tvne and control 
of institution 








2-ycar public 
4-ycar public 
nrivate 
4-year private 
Proprietary 


25.05 
64.22 
1.94 
70.33 
10.35 


6.75 
9.16 
1.36 
10.52 
5.09 


0.270 
0.143 
0.702 
0.150 
0.492 


Dependency status 








Independeni 
Dependent 


UV/.*T*t 

111.45 


8.99 
10.88 


0.149 
0.098 


Type of aid received 








C-B Only 
Pell and C-B 
C-B and GSL 
Pell, C-B and GSL 


29.78 
49.82 
44.52 
47.77 


6.46 
7.29 
8.82 
8.11 


0.217 
0.146 
0.198 
0.170 
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Table 1-3. Estimated sampling errors for mean error per recipient with C-B payment error. 
Instimtional error, $50 tolerance 



Category 


Estimate 


Standard error 


Coefncient 
of variation 


All recipients 


$792.70 


$70.93 


0.090 


Type and control 
of institution 








2-ycar public 
4-ycar public 
2-ycar private 
4-year private 
Proprietary 


$612.91 
$624.37 
$1,084.28 
$1,011.34 
$731.93 


$123.40 
$70.80 
$417.50 
$138.13 
$205.00 


0.201 

Alii 

0.385 
0.137 
0.280 


Dependency status 








Jul W V L/WilU^ ii W 

Dependent 


$791.52 
$793.33 


$101.46 
$84.73"- 


0.128 
0.107 


Type of aid recvved 








C-B Only 
Pell and C-B 
C-BandGSL 
PeU, C-B and GSL 


$826.72 
$621.05 
$930.35 
$822.22 


$121.95 
$85.22 
$207.17 
$127.33 


0.148 
0.137 
0.223 
0.155 
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Tabic J- 1 . Estimated samqpling errors for total C-B distribudonai error: 
Overall sror, $50 tolerance 



Category 


Estimate 
(SmiElions) 


Standard error 
(Smillions) 


Coefficient 
of variation 


All recipients 


$216.49 


$31.20 


0.144 


Type and control 
of institution 








2-year public 
4-ycar public 
2-vear DiivsUc 
4-year private 
Proprietary 


$29.20 
$91.71 
$1.34 
$97.55 
-$3.31 


$15.04 
$20.36 

$3.07 
$23.42 

$7.79 


0.515 
0.222 
2.295 
0.240 
2.353 


Dependency status 








tXiUCpcuucni 

Dependent 


$77.05 
$139.44 


$21.13 
$24.31 


. 0.274 
0.174 


Type of aid received 








C-B Only 
PeUandC-B 
C-BandGSL 
Pell, C-B and GSL 


$48.62 
$49.11 
$102.03 
$16.73 


$10.33 
$20.68 
$20.30 
$12.19 


0.212 
0.421 
0.199 
0.729 



Table J-2. Estimated sampling errors for number of recipients with C-B distributional error 
Overall error, $50 tolerance 



Category 


Estimate 
(thousands) 


Standard error 
(thousands) 


Coefficient 
of variation 


All recipients 


836.44 


28.80 


0.033 


Type and control 
of institution 








2-year public 
4-year public 
2-year private 
4-year private 
Proprietary 


145.29 
379.07 

18.98 
306.93 

36.18 


25.10 
37.01 
8.77 
33.50 
10.76 


0.173 

A nofi 
U.UVo 

0.462 
0.109 
0.297 


Dependency status 








Independent 
Dependent 


298.91 
587.53 


20.46 
24.28 


0.068 
0.041 


Type of aid received 








C-B Only 
Pell and C-B 
C-B and GSL 
PeU, C-B and GST 


108.16 
298.50 
230.05 
249.73 


12.68 
22.45 
24.56 
19.69 


0.117 
0.075 
0.107 
0.079 
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Table J-3. Estimated sampling errors for mean error per recipient with C-B distributional error 
Overall error. $50 tolerance 



Category 


Estimate 


Standard error 


Coefficient 
of variation 


All recipients 


$244.22 


$36.24 


0.148 


Type and control 
of institution 








2-ycar public 
4-year public 
2-year private 
4-year private 
Pioprietary 


$200.98 
$241.95 
$70.51 
$317.82 
.$91.49 


$106.71 
$oj.y2 

$177.12 
$65.27 

$243.46 


0.531 

2.512 
0.205 
2.661 


Dependency status 








Independent 
E}ependent 


$257.77 
$237.33 


$64.56 
$40.63 


0.251 
0.171 


Type of aid received 








C-B Only 
PeUandC-B 
C-BandGSL 
PeU,C-BandGSL 


$449.49 
$164.53 
$443.53 
$66.97 


$90.80 
$69.42 
$71.20 
$51.50 


0.202 
0.422 
0.161 
0.769 
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Table K-1. Estimated sampling errors for total C-B distributional error 
Student eiror, $50 tolerance 



Category 


Estimate 
($miUions) 


Standard error 
($miUions) 


Coefficient 
of variation 


AH recipients 


$127.71 


$29.93 


0.234 


Tvn0 ahA cnntTol 

of institution 








2-year public 
4-year public 

4-year private 
Proprietary 


$15.82 
$62.55 
-$0.14 
$51.71 
-$2.24 


$12.05 
$16.95 

$3.26 
$22.63 

$7.76 


0.762 

0.271 

23.393 

0.438 

3.470 


Dependency status 








Independent 
Dependent 


$58 16 
$69.54 


$20.47 
$23.67 


0.352 
0.340 


Type of aid received 








C-B Only 
PeU and C-B 
C-B and GSL 
PeU, C-B and GSL 


$29.86 
$31.50 
$65.33 
$1.02 


$9.72 
$18.71 
$17.78 
$11.92 


0.326 
0.594 
0.272 
11.688 
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Table K-2. Estimated sampling errors for number of recipients with C-B distributional error 
Student error, $50 tolerance 



Category 


Estimate 
(thousands) 


Standard error 
(thousands) 


CoefHcient 
of variation 


All recipients 


775 5Q 


25.41 


0.033 


Type and control 
of institution 








2-ycar public 
4-ycar public 
2-ycar private 
4-ycar private 
Proprietary 


1 IS IS 

323.69 
18.01 

282.65 
33.05 


20 03 
31.46 

32.29 
10.05 


0.170 
0.097 

0 471 
0.114 
0.304 


Dependency status 








Independent 
Dependent 


254.81 
520.78 


19.09 
22.74 


0.075 
0.044 


Type of aid received 








C-B Only 
Pell and C-B 
C-BandGSL 
PeU, C-B andGSL 


96.74 
255.85 
212.61 
210.40 


12.76 
19.87 
23.76 
15.74 


0.132 
0.078 
0.112 
0.075 
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Table K-3. Estimated sampling errors for mean error per recipient with C-B distributional error. 
Student error, $50 tolerance 





Estimate 


Standard error 


Coefficient 


Category 






of variation 


All recipients 


$164.66 


$38.41 


0.233 



Type and control 
of institution 

2-ycar public 
4-year public 
2-year private 
4-year private 
Proprietary 

Dependency status 

Independent 
Dependent 

Type of aid received 

C-B Only 
Pell and C-B 
C-BandGSL 
PeU, C-^ id GSL 



$133.85 
$193.25 
.$7.74 
$182.95 
-$67.66 



$228.26 
$133.53 



$308.66 
$123.11 
$307.28 
$4.85 



$109.84 
$55.22 

$207.82 
$75.42 

$270.77 



$75.31 
$44.15 



$92.33 
$73.23 
$73.53 
$59.01 



0.821 
0.286 
26.857 
0.412 
4.002 



0.330 
0.331 



0.299 
0.595 
0.239 
12.173 



Table L- 1. Estimated sanq)ling eirors for total C-B distributional error. 
Institutionai error, $50 tolerance 



Category 


Estimate 
(Smillions) 


Standard error 
(Smillions) 


Coefflcient 
of variation 


Alt recipients 


$43.95 


$18.81 


0.428 


Type end control 
of institution 








2-ycar public 
4-ycar public 
2-vear orivate 
4-ycar private 
Proprietary 


$3.43 
$9.90 
$0.20 
$34.33 
-$3.91 


$7.93 
$11.40 

$2.31 
$13.06 

$2.74 


2.310 

I. 153 

II. 457 
0.380 
0.703 


Dependency status 








inucpcnuciii 
Dependent 


$7.79 
$36.17 


$9.32 
$15.30 


1.196 
0.423 


Type of aid received 








C-B Only 
Pell and C-B 
C-BandGSL 
PeU, C-B and GSL 


$21.76 
-$10.05 
$37.02 
-$4.77 


$6.66 
$11.37 
$12.82 

$7.89 


0.306 
1.131 
0.346 
1.652 
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Table L-2. Estimated sampling cirors for number of recipients with distributional error. 
Institutionai error, $50 tolerance 



Category 


Estimate 
(thousands) 


Standard error 
(thousands) 


Coefficient 
of variation 


All recipients 


350.43 


22.53 


0.064 


Type and control 
of institution 








2-ycar public 
4-ycar public 
2-ycar private 
4-ycar private 
Proprietary 


75.51 
153.26 
7.56 
95.40 
18.70 


15.79 
19.00 

3.79 
14.80 

6.84 


0.209 
0.124 
0.502 
0.155 
0.366 


Dependency status 








Tn H^'H^n H^n t 

Dependent 


131.49 
218.94 


12.87 
. 16.20 


0.098 
0.074 


Type of aid received 








C-BOnly 
Pell and C-B 
C-BandGSL 
FeU. C-B and GSL 


32.20 
145.55 

56.71 
115.97 


7.34 
13.66 

9.80 
13.49 


0.228 
0.094 
0.173 
0.116 
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Table L-3. Estimated sampling errors for mean error per recipient with C-B distributional 
Instimtional error, $50 tolerance 



Category 


Estimate 


Standard error 


Coefficient 
of variation 


All recipients 


$125.43 


$55.19 


0.440 


Type and c irol 
of institution 








2-ycar public 
4-ycar public 
2-year private 
4-year private 
Proprietary 


$45.46 
$64.57 
$26.65 
$359.85 
-$208.88 


$107.65 

$384.64 
$140.06 
$173.88 


2.368 
1 183 
14.434 

0.832 


Dependency status 








Independent 
Dependent 


$59.23 
$165.19 


$71.31 
$70.42 


1.204 
0.426 


Type of aid received 








C-B Only 
Pell and C-B 
C-B and GSL 
Pell, C-B and GSL 


$675.59 
-$69.05 
$652.87 
-$41.17 


$153.26 
$73.95 

$185.20 
$66.60 


0.227 
1.071 
0.284 
1.617 
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Table M-l. Estimated sampling enors for total GSL payment 
OvcnOl cnor, $50 tolerance 



Category 


Estimate* 
($milUons) 


Standard erroi* 
(Smillions) 


Ccefflcient 
of variation 


All rtcipients 


$951.05 


$134.61 


0.142 


of institution 








2-year public 
4-year public 

4-ycar private 
Proprietary 


$33.07 
$395.59 

$11.02 
$416.77 

$94.60 


$14.78 
$89.56 
$7.08 
$118.66 
$50.30 


0.447 

0.643 
0.285 
0.532 


Dependency status 








inocpcuociii 
Dependent 


$202.29 
$748.76 


$89.02 
$100.69 


0.440 
0.135 


Typt of aid received 








GSL Only 
Pell and GSL 
C-BaadGSL 
Pell, C-B and GSL 


$702.15 
$71.82 

$142.17 
$34.91 


$121.25 
$17.00 
$31.76 
$5.59 


0.173 
0.237 
0.223 
^^160 



* Due to a rwlalon in the estimate of total GSL loan volume, 
these figures should be reduced by approximately 10 percent. 



^»ui- ui.'i p«rfm.ttd «MsnUn« mors for number of recipients with GSL payment error. 
Overall error, $50 tolerance 



Category 


Estimati^ 
^ (thousands) 


Standard error* 
(thousands) 


Coefficient 
of variation 


All reeipitnts 


721.14 


67.64 


0.094 


A ypW d/M* vWrii#v» 

0/ institution 








2-year public 
4«ycar public 
z-ycoT pny ittv 
4-year private 
Proprietary 


51.42 
370.29 

14.36 
233.28 

51.80 


26.31 
73.20 
10.21 
42.98 
22.58 


0.512 

A 1 QQ 

0.711 
0.184 
0.436 


Dependency status 








inocpcuucui 
Dependent 


104.88 
616.27 


25.52 
66.63 


0.243 
0.108 


I>p« 0/ received 








GSL Only 
PeUandOSL 
C-BandGSL 
PeU,C-BandGSL 


495.33 
71.32 

100.75 
53.74 


60.60 
13.79 
17.30 
7.75 


0.122 
0.193 
0.172 
0.144 



* Due to a r-.- iaion in the estimate of total GSL loan volume, 
these figures should be reduced by approximately 10 percent, 
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Table M-3. Estimated sampling errors for mean error per recipient witii GSL payment error: 
Ovcrail error, $50 tolerance 



Category 


Estimate 


Standard error 


Coefficient 
of variation 


AW recipients 


$1,318.81 


$140.52 


0.107 


Type and control 
of institution 








2-ycar public 
4-ycar public 
2-ycar private 
4-ycar private 
Proprietary 


$643.15 
$1,068.32 

$767.37 
$1,786.62 
$1,826.24 


$140.96 
$110.70 
$370.94 
$364.86 
$500.29 


0.219 

0.483 
0.204 
0.274 


Dependency status 








Independent 
Dependent 


$1,928.85 
$1,214.99 


$619.99 
$101.37 


0.321 
0.083 


Type of aid received 








GSL Only 
PeUandGSL 
C-B and GSL 
PeU, C-BandGSL 


$1,417.54 
$1,006.95 
$1,411.11 
$649.58 


$193.47 
$147.14 
$171.56 
$7';..04 


0.137 
0.146 
0.122 
0.111 



2B8 



Table N-l. Estimated sanwling enors for total GSL payment eiror 
Student eiror, $50 tolerance 



Category 


Estimate* 
($millions) 


Standard erroi* 
(Smillions) 


Coefficient 
of variation 


All rtcipUnts 


$393.52 


$94,22 


0.239 


Typ€ and control 
of institution 








2-year public 
4-year puUic 
2-vear oiivate 
4-year private 
Proprietary 


$16.17 
$116.74 

$10.56 
$194,31 

$55.75 


$9.13 

<A(\ fil 

$7.00 
$77.49 
$36.68 


0,565 
0 350 
0.663 
0.399 
0.658 


Dependency status 








Tndenendent 
DepoKient 


$66.60 
$326.92 


$67.12 
$61.59 


1.008 
0.188 


Type of aid received 








GSL Only 
PeUandGSL 
C-BandGSL 
Pell C-B and GSL 


$277.9-; 
$14,02 
$94.18 
$7.37 


$79.48 
$7,64 

$26,42 
$2,99 


0.286 
0,545 
0.281 
0,405 



* Due to a revision in the estimate of total GSL loan volume, 
these figures should be reduced by approxijnately 10 percent. 
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Tabic N-2. Estimated sampling errors for number of recipients witii GSL payment 
Student enxjt, $50 tolerance 



Category 


Estimate * 
(thousands) 


Standard error* 
(thousands) 


C&«fflcient 
of variation 


AH recipients 


369.42 


54.64 


0.148 


Type and control 
of institution 








2-year public 
4-year public 
2-year private 
4-veamrivate 
Proprietary 


19.39 
140.37 

14.36 
151.40 

43.90 


12.27 
44.00 
10.21 
32.40 
20.98 


0.633 
0.314 

0 711 

0.214 
0.478 


Dependency status 








Independent 


16.06 
353.36 


13.52 
54.82 


0.842 
0.155 


Type of aid received 








GSL Only 
PeUandGSL 
C-BandGSL 
Pell, C-B and GSL 


259.67 
14.49 
82.40 
12.86 


49.70 
6.31 

16.57 
3.54 


0.191 
0.435 
0.201 
0.275 



ERIC 



* Due to a revision in the estimate of total GSL loan volume, 
these figures should be reduced by approximately 10 percent. 
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Table N-3. Estimated sampling errors for mean error per recipient with GSL payment error: 
Stadcat error, $50 tolerance 



Category 


Estimate 


Standard error 


Coefficient 
of variation 


All recipients 


$1,065.24 


$207.68 


0.195 


Type and conUol 
of institution 








2-ycar public 
4-ycar public 

2-ycar private 

* * 

4-year private 
Proprietary 


$833.99 
$831.62 
$735.32 
$1,283.41 
$1,269.89 


$398.03 
$^12.98 
Vi64.85 
$468.98 
$631.96 


0.477 

0.496 
0.365 
0.498 


Dependency status 








Dependent 


$4,147.88 
$925.17 


$2^95.20 
$116.01 


0.553 
0.125 


Type of aid received 








GSL Only 
PeUandGSL 
C-BandGSL 
PeU, C-B and GSL 


$1,070.34 
$968.19 

$1,142.98 
$573.46 


$270.50 
$501.58 
$182.14 
$191.18 


0.253 
0.518 
0.159 
0.333 



Table 0-1. Estimated sampling wrors^to^ 
iistimtionai error, $50 iolwance 



Category 


Estimate * 
(Smillions) 


Standard erroi* 
($millions) 


Coefficient 
of variation 


All recipients 


$627.56 


$106.20 


0.169 


Tvne and control 
of institution 








2-ycar public 
4-year public 

4-year private 
Proprietary 


$29.68 
$318.57 
$0.46 
$239.92 

$38.93 


$8.52 
$83.09 

$0.48 
$88.80 
$26.25 


0.287 

1.049 
0.370 
0.674 


Dependtncy status 








inucpcxiucui 
Dependent 


$139.58 
$487.98 


$63.46 
$85.01 


0.455 
0.174 


T>P* fl/ a<i received 








GSLOnly 
Pell and GSL 
C-BandGSL 
Pell, C-B and GSL 


$467.73 
$71.57 
$56.33 
$31.94 


$100.63 
$15.38 
$14.41 
$4.46 


0.215 
0.215 
0.256 
0.140 



* Due to a revision in the estimate of total GSL loan volume, 
these figures should be reduced by approximately 10 percent. 
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Table 0-2. Estimated sampling errors for number of recipients with GSL payment 
Instiiutional error, 350 iolersncc 



Category 


Estimate* 
(thousands) 


Standard error* 
(thousands) 


Coefflcient 
of variation 


All recipients 


490.15 


70.40 


0.1 V 


Type and control 
of institution 








2-year public 
4-year public 
2-year private 
4-year private 
Proprietary 


49.19 
270.52 
1.88 
147.32 

21.24 


16.53 

1.97 
39.47 
14.96 


0.336 

1.049 
0.268 
0.704 


Dependency status 








Dependent 


90.35 
399.80 


22.63 
63.78 


0.250 
0.160 


Type of aid received 








GSL Only 
PeUandGSL 
C-B and GSL 
PeU,C-BandGSL 


322.77 
67.65 
52.55 
47.18 


62.75 
12.57 
10.63 
7.29 


0.194 
0.186 
0.2a2 
0.155 



* Due to a reviaion in the estimate of total GSL loan volume, 
these figures should be reduced by approxJjnately 10 percent. 
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Table 0-3. Estimated sampling errors for mean error per recipient with GSL payment error: 
Tnctipitionfll mQT. $50 tolerance 



Category 


Estimate 


Standard error 


Coefficient 
of variation 


All recipients 


$1,280.34 


$137.18 


0.107 


Type and control 
of institution 








2-year public 
4-year public 
2-year private 
4-year private 
Proprietary 


$603.45 
$1,177.61 

$244.50 
$1,628.53 
$1,833.12 


$101.96 

CM If 

$115.26 
$0.00 
$403.21 
$423.97 


0.169 

U.UVo 

0.000 

t\ ^ AO 

0.248 
0.231 


Dependency status 








Independent 
Dependent 


$1,544.98 
$1,220.54 


$537.22 
$101.' 


0.348 
0.083 


Type of aid received 








GSL Only 
PeUandGSL 
C-BandGSL 
FeU, C-B and GSL 


$1,449.09 
$1,057.85 
$1,071.92 
$676.97 


$203.01 
$144.07 
$217.38 
$65.18 


0.140 
0.136 
0.203 
0.096 
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UNITED STATES DEPARTMENT OF EDUCATION 
OFFICE OF THE ASSISTANT SECRETARY FOR POSTSECONDARY EDUCATION 



STUDENT 
FINANCIAL ASSISTANCE 
PROGRAMS 



December 9, 1985 



Dear President: 

The Office of Postsecondary Education of the Department of Education 
is conducting a quality control study of the Pell Grant and Campus-Based 
student financial aid programs and the Guaranteed Student Loan 
application certification process* The purpose of the study is to 
determine the amount and type of errors being made in implementing these 
programs, and the probable causes of errors* The study will enable the 
Department of Education to take corrective actions to eliminate or reduce 
these errors* More detailed information on this study can be found in 
the enclosed "Project Summary*" 

Your institution has been randomly selected to participate in the 
study I. and we need specific information from your financial aid 
administrator for the study to be 3-.ocessful* A representative from 
Advanced Technology, Inc*, a research firm located near Washington, D*C*, 
will be contacting your financial- aid eJminist rater in a few weeks to 
schedule a visit, which will last from 1 to 2 days, and will include an 
interview with him or her* Although your financial aid administrator's 
participation in an interview is voluntary, we hope you will urge him or 
her to participate since the results of this study will be used to 
improve the delivery of student financial assistance* 

The interviewer will also verify certain information * ^om a sample of 
approximately 10 students* financial aid files, such as recipient income 
and assets^ and may need to consult with staff from other offices which 
are part of the financial aid process, such as the registrar or bursar. 
Access to your student records is authorized under the Department of 
Education regulations implementing the Title IV programs, 
34 CFR 668* 12(c)(3)* However, since this is a national study, individual 
institutions will not be identified in any reports, nor will 
institutional data be sufficient to make generalizations about individual 
schools in the study. 

This study is being conducted according to the regulations of the 

Privacy Act and Title IV of the Higher Education Act of 1965* The 

interviewer has signed a confidentiality statement under which he or she 

has sv/orn not to reveal to anyone not connected with this study the 




400 MARYLAND AVE.. S.W. WASHINGTON, D.C. 20202 



information obtaintd. If you hava any quastiona about tha atudy or tha 
inatitational viait^ plaaaa contact tha Advancad Tachnology^ Inc^ 
rapraaantativa* Tha toll-fraa talaphona nuinbar, on tha Sprint ayatam, is 
627-2914. 

Thank you in advanca for your aaaiatanca and cooparation. 



Sincaraly# 




Wr^C. Ronald Kinbarling 

Acting Aaaiatant Sacratary 



Enclosure 



278 



UNITED STATES DEPARTMENT OF EDUCATION 

OFFICE OF THE ASSISTANT SECRETARY FOR POSTSECONDARY EDUCATION 



STUDENT 
FINANCIAL ASSISTANCE 
PROGRAMS 



December 9, 1985 



Dear Financial Aid Officer: 

The Office of Postsecondary Education of the Department of Education 
is conducting a study of the Pell Grant and Campus-Based student 
financial aid programs and the Guaranteed Student Loan application 
certification process. The purpose of the study is to determine the 
amount and type of errors being made in implementing these programs, and 
the probable causes of errors. The study will enable the Department of 
Education to take corrective actions to eliminate or reduce these 
errors. More detailed information on this study can be found m the 
enclosed "Project Summary." 

Your institution has been randomly selected to participate in the 
study, and we need specific information from you for the study to be 
successful. A representative from Advanced Technology, Inc., a rosearch 
firm located nea*- Washington, D.C., will be contacting you in early 
January to scheduie a visit, which will last 1 to 2 days, and will incude 
an interview with you. Although your participation in the interview is 
voluntary, we urge you to participate since the results of this study 
will be used to improve the delivery of student financial assistance. 
When the interviewer calls you, we would like to collect some preliminary 
information about the number of recii;>ients in each program at your 
institution. We will need an unduplicated count of recipients m the 
Campus-Based programs (NDSL, SEOG, and CW-S), a count of Pell Grant 
recipients and a count of students certified to receive Guaranteed 
Student Loans. 

Your interviewer will also arrange to verify certain information from 
a sample cf approximately 10 students' files, such as recipient income 
and assets, and may need to consult with staff from other offices which 
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400 MARYLAND AVE.. S.W. WASHINGTON. D.C 20202 



are a part of the financial aid process^ such as the registrar or 
bursar. Tno enelosed pags lists what information ^ill bs needed nhen ne 
call to schedule the visit, as well as items the interviewer will need to 
obtain at the time of the visit. 

Access to your student records is authorized under Department of 
Education regulations implementing the Title IV programs, 34 CFR 
668. 12(c) (3}. However, since this is a national study individual 
institutions will not be identified in anj- reports / nor will 
institutional data be sufficient to make generalizations about individual 
schools in the study. 

This study is being conducted according to the regulations of the 
Privacy Act and Title IV of the Higher Education Act of 1965. The 
interviewer has signed a confidentiality statement under which he or she 
has sworn not to reveal to anyone not connected with this study any 
information you provide. If you have any questions about the study or 
the institutional visit, please contact the Advanced Technology l.ic., 
representative. The toll-free telephone number, on the Sprint system is 
627-2914. 

Thank you in advance for your assistance and cooperation. 



Sincerely, 




Ernst Becker, Director 
Division of Quality Assurance 



Enclosures 
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PROaECT SUMMARY 



TITLE IV QUALITY CONTROL PROJECT 

STAGE II: AN INTEGRATED PROJECT 



Background 

The Title IV Student Financial Assistance programs have grown dramatically in both 
dollar volume and student participation during the past decade* With this rapid growth 
has come the potential for errcrs in student application information, student eligibility 
certification, award calculations, and other program procedures. The Department of 
Education is increasingly aware of the need to reduce these errors and to improve 
performance in all Federal student assistance programs. The Department is committed to 
ensuring that thc^e programs operate efficiently and that funds are allocated properly. 

Project Objectives and Activities 

The two stages of the Title IV Quality Control Project are designed to measure the 
degree to which errors exist in the delivery of all five major sources of student aid, 
identify causes of error, recommend management corrective actions, and provide techni- 
cal assistance and follow-up analysis. Stage I was a pilot study to determine the 
feasibility of measuring error in the Campus-Based programs and the Guaranteed Student 
Loan (GSL) certification process. Included in the pilot were th^ National Direct Student 
Loan (NDSL), Supplemental Educational Opportunity Grant (SEOG), and College Work- 
Study (CWS) programs, known as the "Campus-Based" programs, and the Guaranteed 
Student Loan program. In Stage II the scope has been expanded to include all five major 
Title IV student assistance programs by adding the Pell Grant program. 

The primary activities conducted during Stage I (198^) included the following: 

• The 1983-8* program-wide error rates by number of cases and amount of 
dollars for the Campus-Based and GSL programs were document<*d, computed, 
and analyzed. 

e Institutional compliance with Federal legislation and regulations, school- 
specif :c packaging philosophies, need analysis principles, and other school 
administrative policies and procedures were documented and analyzed. 

• The major types of program errors were identified and analyzed. 

• The effectiveness of quality control procedures and corrective actions which 
had already hfeen implemented to reduce or prevent irror was evaluated. 

• Recommendations were developed for management actions to correct each of 
the major errors identified. 

These activities were based on data gathered through a nationwide survey of 820 
students and their parents, and 281 postsecondary institutions in the spring of 198*. 
Trained interviewers visited a random, representative sample of public, private, and 
proprietary Institutions* At each Institution, the interviewer selected a random sample of 
Campus-Based and GSL recipients and reviewed their financial aid records. The financial 
aid administrator was also interviewed and asked to describe the institution's student aid 
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awarding procedures. Another group of experienced Interviewers visited the students who 
were selected, and their parents, and asked them to supply documents verifying the 
Information that appeared on their application forms. 

Findings from Stage I suggest that error can be quantified and Is significant In the 
Campus-Based program and GSL certification process. Based on these findings, a broader 
effort will be undertaken in Stage II to more precisely define and measire error, and 
obtain a more comprehensive understanding of Pell, Campus-Based, and GSL program 
error. In Stage II, the 1985-86 program-wide error rate for the five programs will be 
measured, probable causes for major types of error will be Identified, and recommenda<* 
tlons for corrective actions will be developed. In particular, the Pell error rate will be re- 
measured to determine the effectiveness of recently expanded validation requirements. 
Stage II will also evaluate the effectiveness of other Institutional quality control 
procedures In reducing payment error. 

Stage II methodology will be the same as In Stage I: a representative sample (about 
300) Institutions will be visited; student record Information collected; financial aid 
administrators Interviewed; and students (about 3,000) and their parents asked to provide 
documents verifying their application Information. 

Project Contractor 

The Office of Student Fi^^anclal Assistance awarded the Title IV Quality Control 
Project contract In January ISi* to Advanced Technology, Inc., of Reston, Virginia, and 
its subcontractor, Westat, Inc., of Rockvllle, Maryland. Interviewers who visit institutions 
are employees of Advanced Technology; Interviewers who visit students and their parents 
are employees of Westat. Ms. Carol Miller of Advanced Technology Is the project 
director. Mr. Robert Lear month Is the project team leader for Westat. 

Sponsoring Agency 

The Title IV Quality Control Project Is sponsored by the Statistical Analysis Branch 
of the Division of Quality' Assurance, Debt Collection and Management Assistance 
Service, U.S. Department of Education. Dr. David Iwamoto Is the chief of the Statistical 
Analysis Branch, and Mr. Ernst Becker Is the director of the Division of Quality 
Assurance. 

Additional Information may be obtained from Ms. Karen Chauvin, project officer for 
the Statistical Analysis Branch, Division of Quality Assurance at (202) 2*5-0102. 
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i 
















1 














1 

% 


X 
p 

1 

1 










DC 




1 
1 


^ 


IDi 


SCHOOL NME 


t 

DATE CAYS 

. t _ . - , ! '. 


COWENTS 


i 


i- 


05-M 


BRICK CfflPUTER SCIENCE INSTirjTE 


- : 1 

1 1 
) 1 


1 

? 1 






I g 

! -f 






GRANT HOSP SCH OF NURSING 


I 1 




1 


OUT OF Bl-SINESS 




09-03 


SPECIAL THAWING OPPORrjNITY PRSM 


1 i 




1 


NON LCCATABLE 


t -1 




16-04 


HI-fASHION BEAUTY COLLEGE 


1 i 




1 


NONLOCATAELE 


1 vS 




06-11 


UASHINBTON HOSP CTR SCH OF NURSING 


1 1 




1 


OUT OF BUSINESS 


1 ^ 




04-07 


SCHUVLER-OCMJNG-TIOGA SCH PRAC NUR 


1 1 




1 


NONLOCATAKJE 


1 




05-02 


PRINCETON NED CTR SCH OF ICD-TECH 


1 1 




1 


OUT 0FMMKEftl4tMCC5 






10-10 


ST FIOELIS COLLEGE 


i 1 




1 


OUT OF BUSINESS 






£5-16 


CRARQN Hia COLLEGE 


1 i 




1 


OS/r OF SAWIE 






19-09 


RICH ( JOES ART SCH OF HAIR I)ESIQ< 


1 1 




1 


OUT OF BUSINESS 


1 


i- 


01-11 


KOOALY MUSICAL TRAINING INSTITUTE 


I 1 




1 


NON LOCAfABLE 




19-07" 


EAST TEXAS BAPTIST COLLEGE 


1 2/10 1 


2 1 


BARNES 1 




1 4 




19-06 


LETOUffCAU COLLEGE* 


1 2/12 1 


2 


BAffCS i 








19-08 


KILGORE COLLEGE 


1 2/17 1 


2 


BARNES 1 








l»-04 


NORTH TEXAS STATE UNIVERSITY 


1 2/19 ! 


3 


BARNES 1 




1 Sir 




19-03 


UNIV OF HAIR DESIGN 


1 2/24 1 


2 


BARTCS 1 








19-05 


COOKE COUNTY COLLEGE 


1 2/26 i 


2 


BARNES 1 








19-02 


VOGUE BEAUTY COLLEGE kStll 


1 2/2d 1 


4 


BARNES 1 


COMBINED VISIT 






22-01 


VOGUE BEAUTY COLLEGE «11 


i 3/03 1 


2 


BARfCS 1 




1 




19-01 


UESTERN OKLAHOMA AREA VX TECH CTR 


i 3/06 i 


2 


1 BARNES 1 




1 S 




19-12 


UNITED TEQWICAL INSTITUTE 


1 3/13 1 


2 


1 BARNES 1 




1 5 

1 




19-10 


ROGERS STATE COLLEGE 


1 3/17 1 


2 


1 BAI^S 1 




I 




ig-13 


MfTH OKLflHOMO PTTV .10 m 1 FRF 






1 mms 1 




1 




24-05 


GOLDEN GATE UNIVERSITY 


1 2/10 1 


2 


1 BARON 1 




1 1 




24-03 


SAN FRANCISCO COL OF MORTUARY SCI 


1 2/12 1 


1 


1 BARON 1 




1 




24-06 


SAN FRANCISCO CNTY aS CENTERS 


1 2/ld i 


2 


1 BARON 1 








24-07 


ACADEMY OF STENOGRAPHIC ARTS* 


i 2/20 i 


2 


1 BARON 1 




1 '1 




24-08 


SAN JOSE' STATE UNIVERSITY 


1 2/24 1 


2 


1' BARON 1 




1 ^ 






VMPI 


WW W^m^ 




1 ^ 




24-10 


UNIVERSITY OF THE PACIFIC 


1 3/03 1 


2 


1 BARON 1 




1 i 




24-12 


CONTRA COSTA COLLEGE 


1 3/05 1 


1 


1 BARON 1 




1 

1 z 




24-04 


COGSWELL COLLEGE 


1 3/06 1 


2 


1 BARON 1 


• 


1 

1 < 




24-01 


SANTA ffiSA JUNIOR COLLEGE 


1 3/10 1 


2 


1 - BARON 1 








24-02 


AMERICAN RIVER COLLEGE 


1 3/lS 1 


3 


1 BARON i 




•i- 




25-02 


^CNTURA COUNTY COMMITY DISTRICT 


1 3/17 1 


3 


1 BARON 1 








as-oi 


SAUYER COLLEGE AT VENTURA 


1 3/20 1 


2 


1 BARON 1 




\ t 




21-09 


KANSAS STATE UNIICKITY 


1 2/24 1 


5 


1 BUSHAU 1 




1 * ^ 




17-12 


MONTANA STATE UNMRSITY 


1 3/11 1 


4 


1 BUSHAU 1 




1 1 


\ 


17-U 


CARROLL COLLEBE 


1 3/17 1 


2 


U^BUSHAN 1 
r Btnravi 1 






23-U 


COLLESE OF THE REDUOOOS 




2 




1 




10-11 


BUTLER COUNTY COmJNITY COLLEGE 


1 2/17 1 


2 


1 BUSSE 1 




' 




10-08 


SHENANGO VALLEY SGPOOL GF BUSINESS 


1 2/19 1 


2 


1 BUSSE 1 








07-04 


HuLLIMB GULEGE 


1 3/03 1 


2 


1 BIJSSE 1 




I i 


i 


10-12 


UNIVERSITY OF PIH-MAIN 


1 3/10 1 


5 


1 BUSSE . 1 








S-06 


SOUnCRN CALIFORNIA SCHOOL OF THEOL 


1 1 


2 


1 CWISTENSEN 1 








25-10 


L06 ANGELES CLB OF CHIROPRACTIC 


1 2/10 1 


2 


1 CHRISTENSEN 1 




1 




25-11 


RIO HOWO CQLLIGE 


1 2/12 1 


2 


1 CHRISTENSEN 1 




1 "1 


i 


25-17 


CYPRESS COLLEGE 


1 2/18 1 


1 


1 CHRISTENSEN 1 




1 4 




25-12 


UNI\^ITY OF CALIFORNIA-LAt 


1 2/19 1 


3 


1 CHRISTENSEN 1 




' 1 

t 


f 


25-04 


ST JO»B SEMINARY 


1 2/25 1 


1 


1 CHRISTENSEN 1 






25-06 P0(MACaiE6E 


1 2/26 1 


2 


1 CHRISTENSEN 1 




1 






D 
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101 SCHOOL NME 



25-09 CERRITOS COLLEGE 

25-15 SAN BERNARDINO VMIEY COLLEGE 

25-U SAN BERNARDINO CTY MED CTR l€D TECH 

25-03 flMTarPE VALLEY COLLEGE 

25-05 PITZERCOLEGE 

25-13 VICTOR VALLEY BEAUTY COLLEGE 

25-18 SAN DIEGO STATE UNIVERSITY . 

25-19 US imERNATIONRL UNIVERSITY 

25-07 CLARENONT tCKBtlA COLLEGE 



DATE 

i 2/28 I 
I 3/03 I 
i 3/05 I 
I 3/06 I 
I 3/10 I 
I 3/13 I 
I 3/17 I 
I 3/25 I 
: 3/27 I 



i 

DAYS 
1. 

1 I 

2 I 
1 I 



DC 

NAME 



COMENTS 



GHRISTENSEN 
CHRISTENSEN 
CHRI3TENSEN 
CHRISTENSEN 
CHRISTENSEN 
CHRISTENSEN 
CHRISTENSEN 
CHRISTENSEN 
CHRISTENSEN 



06-06 UNIVERSITY DISTRICT OF COLUMBIA* 12/111 2 1 C.SWTH 

06-12 ORGE MASON IWVERSITY I 2/13 I 2 1 C SMITH 

05-05 USINUS CttLFfiF I 3/12 I 2 I C.SWITH 

0»-09 MOWMK VALLEY COWUNITY CL6 I 3/19 I 2 I C.SNITH 



02-Oi UNIVEI6ITY OT CONNECTIOJT-STORRS I 2/10 I 3 1 FARRELL 

02-06 UNIVERSITY OF BRIDGEPORT I 2/13 I 2 1 FARRELL 

02-04 YALE UHI«RSITY* I 2/17 I 3 1 FARRELL 

02-05 COUNTY SCHOOLS HONE STUDY I 2/20 I 2 1 FARRELL 

02-02 BRYANT COLEGE I 2/2* I 2 I FARREU 

02-03 BSOWN UNIVERSITY I 2/2S I 2 1 FARREIl 

02-09 PACE UNIVERSITY I 3/03 I 2 I FARRHi. 

02-10 SWY COLLEGE-PURCHASE I 3/05 I 2 1 FARRELL 

02-12 RUT6ERS-STATE IWV NEU JERSEY I 3/12 I 3 1 FARRFIL 

02-11 HONTOfllR STATE COLLEGE I 3/17 I 3 I FflRfftLL 

02-08 ICRCV COLLEGE I 3/20 I 2 1 FARRELL 

04-11 LEMOYNE COLLEGE I 3/2* I 2 I FPRRELL 

fti-ni .aiivmiFBF-BimpnRT !J/ ?g I 3 I FARRELL 



18-08 CONCORDIA COLLEGE . I 2/10 I 2 I FLOYD 

18-07 UNIVERSITY OF UISC-CTRL SYSTEM I 2/12 I 3 I FLOYD 

18-13 NADISON AREA TECHNICAL COLLEGE I 2/17 I 2 1 FLOYD 

18-06 UNIVERSITY OF KISC-MMISON* I 2/19 I 2 1 FLOYD 

18-09 MARIAN COLLEGE OF FOND DU LAC I 2/24 I 2 I FLOYD 

18-10 UNIVERSITY OF MISC-OSHKOSH I 2/26 I 3 1 FLOYD 

18-11 UIKESHORE BD-VOC/TECH-AOULT EDUC I 3/03 I 2 1 FLOYD 

18-12 UNIVERSITY OF aiSC-STEVENS POINT I 3/05 ! 3 1 FLOfO 

18-05 SOUTHWEST WISCONSIN VOC TECH INST I 3/JO I 2 I FLOYD 

18-04 UNIVERSITY OF yiSC-PLATTEVILLE I 3/12 I 2 1 FLOYD 

18-03 KIRKUOOD COMIMITY COLLEGE I 3/17 I 2 I FLOYD 

18-01 lOUA STATE UNIV OF SCIENCE 4 TECH I 3/19 I 3 1 . FLOYD 

Ifl-Qg SIWSCN COLlfg _l 3^ I 2 I&»>1MWL 



09-02 BflOURRD CO»IU»TY COLLEGE 

09-04 PALM BEACH JR COLLEGE 

09-OQ IWIAN RIVER COmU^ilTy CaiEGE 

09-07 IWVERSITY OF FLORIDA* 

09-06 SANTA FE COMMUNITY COLLEGE 

09-09 FLORIDA A ft N UNIVERSITY 

09-08 FLORIDA STA1I DIVERSITY 

09-10 THOMAS AREA TECHNICAL SCH 

09-11 ABRAHAM QALOUIN AGRICULTURAL 

09-12 BEN HILL-IRUIN AREA VOC TECH 

09-01 MIAMI DADE CONUNITY COLLEGE 

14-03 JACKSONVILLE STATE IWIVERSITY 



CLG 



I 2/10 I 

I 2/13 I 

I 2/17 I 

I 2/19 I 

I 2/24 I 

I 2/27 I 

I 3/03 I 

I 3/06 I 

I 3/10 I 

I 3/13 I 

I 3/17 I 

I 3/26 I 



2 I 

2 I 
1 I 

3 I 
2 

2 
2 
2 
3 
2 
5 
3 



FREEOMAN 
FREEDHflN 
FREEDMAN 
FREEDMAN 
FREEDMAN 
FREEDMW 
FREEDMAN 
FREEDMW 
FREEDMAN 
FREEDMAN 
FREEDMAN 
FREEDMAN 
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IM SCHOOL NflIC 



U-04 AUBURN UNIVERSITY 

ttlBUHN 'JNIVERSITY-WNTGOWERY 



DATE DAYS 

3/31 I 3 
V03 I 2. 



DC 

NAME 



COtTEHTS 



20-01 AUSTIN COWUNITY COLLEGE* I 2/10 I 2 

20-13 UNtV OF HOUSTON/DOUNTOUN aS I 2/13 I 2 

20-03 PAN AftRICflN UNIVERSITY 1 2/18 I 2 

20-05 SOUTH TEXAS VOCATIONAL TECH INST I 2/20 1 2 

20-04 TEXAS STATE TECH INST~RID GRANDE I 2/24 I 2 

20-02 TEXAS A t I UNIV 1 2/26 I 2 

20-08 SOUTHERN TEOKICAL CLB OF LAFAYETTE I 3/03 I 2 

20-09 UNIVERSITY SOUTHMESTERN LA I 3/05 I 3 

20-10 SOUTfCRN UNIVERSITY A I « COLLEGE I 3/10 I 3 

20-11 SOUTHEASTERN LOUISIANA UNIV I 3/13 I 2 

20-12 BIGGERS SCH COURT RPT/SECRETAR SCI I 3/17 I 1 

Pft-flft MfljSFY BUSItfSS atLEGE ] \ ? 



12-06 UNIVERSITY OF DAYTON 


1 2/10 1 


2 1 


GILWUR 1 ■ 


12-05 MIANI UNIVERSITY* 


1 2/12 1 


2 1 


GILMOUR 1 ' 


12-01 INDimA CENTRAL UNIVERSITY 


1 2/17 1 


2 1 


GILMOUR 1 ' 


12-02 J. EVEREn' LIGHT CAREER CENTER 


1 2/19 1 


2 ! 


GILMOUR 1 1 


12-03 m TECmiCAL INST-IM)IANAPOLIS 


1 3/24 1 


3 1 


GILMOUR 1 1 


12-10 UNIVERSITY OF KENTUCKY 


1 3/03 1 


3 1 


GILMOUR 1 > 


12-12 EASTERN KENTUCKY UNIVERSITY 


1 3/06 1 


2 1 


GILMOUR 1 ' 


12-11 CENTRAL KENTUCKY STATE VOC TECH 


1 3/10 1 


2 1 


GILMOUR 1 > 


12-09 SHAt»K STATE CONNUNITY COLLEGE 


1 3/13 1 


2 1 


GILMOUR 1 < 


12-Oe MARIEHA COLLEGE 


1 3/17 1 


2 1 


GIIMUR 1 ' 


12-07 OHIO UNIVERSITY 


1 3/19 1 


3 1 


GILMOUR 1 1 


08-U ttCRICm COLLEGE FDR APPLIED ARTS 


1 2/19 1 


1 1 


GOLDSMITH 1 ' 


08-15 CHAnANOOGfl STATE TECH CCN CLS 


! 2/20 1 


2 1 


eCLDSHITH 1 — 1 


U-OG UaOTECH UELDIN6 EDUCATION CENTER* 


1 2/12 1 


2 1- 


HOYT 1 


11-07 HIGHAND PARK COWUNITY COLLEGE 


1 2/17 1 


2 1 


HOYT 1 


11-08 OAKLAND COH(UNITY COLLEGE 


1 2/19 1 


2 1 


HOYT 1 


11-09 UNIVERSITY OF MICHIGAN 


1 2/24 1 


2 1 


HOYT 1 


11-10 ALBION COLLEGE 


1 2/27 1 


2 1 


HOYT 1 


11-04 BWER JUNIOR COLLEGE OF BUSINESS 


! 3/04 1 


2 1 


HOYT 1 


11-03 SA6INAU BUSINESS INSTITUTE 


1 3/06 1 


2 1 


HOYT 1 


n-02 NORTHUOOO INSTITUTE 


1 3/10 1 


2 1 


HOYT 1 


11-01 KIRTLAND COIHJNITY COLLEGE 


1 3/12 1 


w i 


HOYT 1 


11-12 CALVIN COLLEGE 


1 3/17 1 


3 1 


HOYT 1 


11-11 GRAM) RAPIDS SCH-BIBLE i MUSIC 


1 3/20 1 


2 1 


HOYT 1 — 


15-07 UNIV ILLINOIS-CHICAGO CIRaE CAKPUS 


1 2/10 1 


2 1 


JACKSON 1 


17-05 BRAINERD COMMUNITY COLLEGE 


1 2/13 1 


2 1 


JACKSON 1 


15-10 MORAIIC VALLEY COmUNITY COUEGE* 


1 2/18 1 


2 1 


JACKSON 1 


15-13 DOREE SCHOOL OF BEAUTY CULTURE 


1 2/20 1 


2 1 


JACKSON 1 


15-12 DEVRY INSTITUTE OF TECHNOLOGY 


1 2/^ 


2 1 


JACKSON 1 


15-11 ILLINOIS INSTITUTE OF TECH 


1 i/26 1 


2 1 


JACKSON 1 


15-04 NATIONAL COLLEGE OF EDUCATION 


1 3/03 1 


2 1 


JACKSON 1 


15-05 NORTHUESTERN UNIV 


1 3/05 1 


2 1 


JACKSON 1 


15-03 ST FRANCIS HOSP SCH OF NURSING 


1 3/07 1 


1 1 


JACKSON 1 


15^ UilVERSITY OF WISC-PARKSIDE 


1 3/10 1 


2 1 


JACKSON 1 


15-01 SATEUAY TE3HICAL INSTITUTE 


1 3/12 1 


2 1 


JACKSON 1 


15-08 CITY COLLEGE OF CHICAGO 


1 3/17 1 


3 1 


JACKSON 1 


15-09 TRINITY CHRISTIAN COLLEGE 


1 3/20 1 


2 1 


JACKSON 1 



FfCEDMAN 



GAMBLE 
GAMBLE 
GAMBLE 
GAMBLE 
GAMBLE 
GAMBLE 
GAMBLE 
GAMBLE 
GAMBLE 
GAMBLE 
GAMBLE 
GflWLE 



ERIC 



D-9 



2^5 



lOt SCHOOL NAME 



DATE 



U-07 NORTHMEST MISSISSIPPI JR CLG 

U-06 PHILLIPS COLTfTY COMMUNITY COLLEGE 

14-03 BLUE NOUNTfllN COLLEGE 

l*-Od RUSTCOLLtSE 



i 

DAYS 
1- 

I 3/26 I 2 I 
I 3/31 I 2 I 
I MM. I 2 I 



DC 

NflME 

JACXSON 
JACKSON 
JPCKSON 
JACKSON 



17-04 FERGUS FALLS ST HOSP CHEM DEP CTP 

17-02 NOORHEflO STATE UNIVERSITY 

04-02 SmV AT BIFFALO 

04-12 ERIE COMMUNITY COLLEGE-NORTH CflMPUS 

17-Oi CONCORDIA COLLEGE-MOORtCAO 

17-06 UNIVERSITY OF Mim-MINNEAPaiS 

17-07 ST OLflF COLLEGE 

17-08 GOGEBIC COMMUNITY COLLEGE 

17-10 NORTHERN MICHIGAN UNIVERSITY 

17-09 SUOMI COLLEGE 



I 2/21 I 1 I 
I 2/24 I 3 I 

I 
I 



I 2/27 I 

I 3/03 ! 

I 3/10 I 

I 3/13 I 

I 3/17 I 

I 3/20 I 



JOHNSON 
JOHNSON 
JOHNSON 
JOHNSON 
JOHNSON 
JOKSON 
JOHNSON 
JMm 
JOHNSON 
JOtffiON 



COtCNTS 



05-09 UNIVERSITY OF POMSYLVANIflf I 2/10 I 3 1 JAMES 

05-10 PHILflOaPHIA CLG-PHAMACY I SCIENCE I 2/13 I 2 I JAMES 

05-13 CAMDEN COUNTY COLLEGE I 2/17 I 2 I JAMES 

05-12 P B METHOD OF HAIR DESIQi I 2/19 I 2 I JAMES 

05-14 6LAS3S0R0 STATE COLLEGE I 2/24 I 3 I JAKS 

14-14 PHILLIPS COLLEGE I 2/27 I 2 I JAfCS 

05-07 VALLEY FORGE CHRISTIAN COLLEGE I 3/17 I 2 I JAMES 

05-06 MONTGDCRY COUNTY COMMUNITY COLLEGE I 3/19 I 2 I JAKS 

05-08 60R00N PHILLIPS SCH OF BTY CULTURE I 3/21 I 1 I JAMES 



03- 03 NASSAU COMMUNITY COLLEGE I 2/10 I 

04- 14 COMMUNITY aG OF FINGER LAKES I 2/12 I 
03-12 GENERAL T)€QL06ICAL SEMINARY* I 2/14 I 
03-04 CITY UNIV OF NEU YORK-CENTRPL I 2/17 I 
03-02 ADELPHI' UNIVERSITY I 2/26 I 
03-09 TEACHERS COLLEGE-COLUMBIA UNIV I 3/03 I 
03-01 STEVENS INSTIL OF TECHNOLOGY I 3/06 I 
03-07 COMMERCIAL PRGGRAWING UNLIMITED I 3/10 I 
03-10 ST JOfi'S UNIVERSITY I 3/11 I 
03-OS NANHArAN TECrtllCAL INSTITUTE I 3/13 I 
03-06 ROYAL BUSINESS SCHOOL I 3/17 I 
03-08 COLUMBIA UNIVERSITY I 3/19 I 



1 I 
7 I 

2 I- 
2 I 
2 I 

1 I 

2 I 
2 I 

2 I 

3 I 



J. SMITH 
J.SMITH 
J.SMITH 
J.SMITH 
J. SMITH 
J.SMITH 
J.SMITH 
J.SMITH 
J.SMITH 
J.SHITH 
J.SMITH 
J.SMITH 



10-03 r€IDEL£ER6 COLLEGE* I 2/10 I 2 I KINSEL 

10- 05 MANSFIELD BUSINESS COLLEGE I 2/13 I 2 I KiNSa 

11- 05 MERCY COLLEGE OF DETROIT I 2/17 I 2 I KINSEL 
17-03 DETROIT LAKES AREA VOC TECH INST I 2/19 I 2 I KINSEL 
10-01 CLEVELAND INSTITUTE OF ART I 2/24 I 2 I KINSEL 
10-02 CASE UESTERN RESERVE WIV I 2/26 I 3 1 KINSEL 
10-09 SLIPPERY ROCK UNIVERSITY I 3/03 I 3 I KINSEL 
10-07 PA ACAO OF C0SMTL6Y ARTS t SCI I 3/06 I 2 I KINSEL 
04-06 STATE UNIVERSITY COLLEGE AT CtCONTA I 3/11 I 2 I KINSEL 
04-10 HERKIMER COUNTY COMMUNITY COLLEGE I 3/13 I 2 I KINSEL 
04-03 SONY C0LL6 OF TECH I 3/17 I 2 I KINSa 
10-06 IWIAWfl UNIVERSITY OF PENNSYLVANIA I 3/19 I 3 I KINSEh 



06-12 CLAFLIN COLLEGE 

06-11 SOUTH CAROLINA STATE COLLEGE 

OB-06 FRANCIS MARION COLLEGE 

08-10 MORRIS COLLEGE* 



i 2/10 
I 2/12 
I 2/17 
I 2/20 



2 I 
2 I 
2 I 
2 I 



ERIC 



D-IO 



LAN6LEY 
LAN6LEY 
LAN6LEY 
LANGLEY 

2«6 



DC 



ID! 


SCHOOL NAME 


Dfl'E 
— 1 1. 


DAYS 
i— 


iwinc 


03-13 


MiDLflNDS 'tCffllCflL CGL'.EGE 


1 ZlZh 1 


2 ! 


LANGLEY 


08-09 


SOUMflS^SN COWRJNITY CCLLEGE 


1 2/26 1 


2 1 


LAN6LEY 


oa-07 


'university of N C-WlLKI^iGTCN 


1 3/03 ! 


2 1 


LANGLEY 


Ofl-04 


SHM UNIVERSITY 


I 3/06 1 


2 1 


LK«LEY 


08-06 


WAKE TECHNICAL COLLEEE 


1 3/10 1 


1 1 


LANGLEY 


08-05 


ST flUGUSTINES COLLEGE 


1 3/11 1 


2 1 


LANGLEY 


08-03 


UTLKES COMNIMITY COLLEGE 


1 3/'#V 


1 1 


LANGLEY 


08-01 


flPPflLflCHIflN STATE UNIVERSITY 


1 3/17 1 


3 1 


LANGLEY 




LEES NCRAE COLLEGE 


1 3/20 1 


2 1 


LANGLEY 


01-06 


SORDON COLLEGE 


1 2/20 1 


2 1 


LEIBMN 


05-01 


PRINCETON UNIVERSITY 


1 3/03 1 


3 i 


LEIBNAN 


05-03 


MESTWNS'iER CHOIR COLLEGE 


1 3/06 ! 


2 1 


LEIBMAN 


01-U 


BOSTON UNIVERSITY 


1 3/24 ! 


3 1 


LEIBNM 



COWENTS 



06-03 NORTHERN VA CNTY CLG* I 2/2^ I 2 

06-09 WCRICAH UNIVERSITY I 2/27 I 2 

06-08 CATHOLIC UNIVERSITY OF AlCRICA 1 3/03 1 2 

06-07 SOiniBSTERN UNIVERSITY I 3/05 I 2 

06-10 BENJANIN FRANKLIN UNIVERSITY I I 2 

06-05 COPPIN STATE COLLEGE I 3/10 I 2 

06-03 JOHNS HOPKINS UNIVERSITY I 3/12 I 2 

06-04 COMtHm COLLEGE OF BALTIMORE I 3/19 I 3 

03- 11 MTI BUSItCSS SCHOOL I 3/24 I 2 

04- 13 KEUKACaLEGE I 3/26 I 2 



MACK 
MACK 
MACK 
MACK 
MACK 
MACK 
MACK 
MACK 
JWCK 
MACK 



13-04 
13-07 
13-05 
13-12 
13-11 
13-01 
13-02 

13- 09 

14- 13 



TEUCSSEE STATE IWIVERSITY I 2/13 I 3 

DAVID LIPSCOW COLLEGE I 2/17 I 3 

MIDDLE TENNESSEE STATE UNIV» I 2/20 I ^ 

JACKSON STATE COMMIITY COLLEGE 1 2/24 I 2 

UEST TENNESSEE BUSINESS COLLEGE I 2/26 I 2 

0HEN5BOR0 AREA VOC EDUCATION CTR I 3/03 I 1 

KENTUCKY ICSLEYflN COLLEGE I 3/04 I 2 

mim CITY COLLEGE I 3/06 I 2 

HINDS JUNIOR COLLEK I 3/11 I 2 



RENSCHLER 
RENSCHLER 
RENSOIER 
RENSCHLER 
RENSCHLER 
RENSOIER 
RENSOIER 
RENSOtxR 
RENSCHLER 



07-07 JOHN TYLER COMMUNITY CttLEGE 

07-06 ST PAULS COLLEGE 

07-05 ROANOKE lO HOSP 5CH OF NURSING* 

07-08 TIOEUATER COMUilTY COLLEGE 

07-02 NEU RIVER COWWITY CaLEGE 

07-03 UYTHEVILLE COWUNITY COLLEGE 

04-05 SUNY AT BINGHANTON 

04-04 SUNY AGRICULTURAL t TECH CaLEGE 

07-09 COmONUEALTH COLLEGE 

07-01 UEST VIRGINIA HESLEYAN COUEGE 

06-01 PEtXSYLVRNIP STATE UNIV 



I 2/10 I 

i 2/12 I 

I 2/17 I 

I 2/20 1 

I 3/34 I 

I 3/36 I 

I 3/03 I 

I 3/06 I 

I 3/10 I 

I 3/13 I 

I 3/17 I 



ROSS 
ROSS 
ROSS 
ROSS 
ROSS 
ROSS 
ROSS 
ROSS 
ROSS 
ROSS 
ROSS 



33-05 


WHITMAN COLLEGE 


1 3/11 1 


3 1 


STALLCOP 


33-06 


UALLA MLLA COLLEGE 


1 3/13 1 


3 1 


STAUCOP 


33-03 


J « PERRY INSTITUTE* 


1 3/18 1 


3 1 


STALLCOP 


33-04 


YAKIMA VALLEY COLLEGE 


1 3/30 1 


3 1 


STALLCOP 


33-08 


PORTLAND COMMUNITY COLLEGE 


1 3/24 1 


3 1 


STALLCOP 


23-07 


LEUIS t CLARK COLLEGE 


1 3/37 1 


3 1 


STALLCOP 


33-01 


UNIVERSITY OF UASHIN6T0N 


1 3/03 1 


3 1 


STALLCOP 


^23-02 


UNIVERSITY OF PUGET SOUND 


1 3/06 1 


3 1 


STALLCOP 
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IM SCHOOL NAIC 



23-10 HUMOLDT STflTt UNIVERSITY 

23-09 PWWNS TIGftRD BEflUTY SCHOtt 

17-13 iMTCflNftTKlW INST OF HfllR DESIGN 

3?-n TPrmiCHL IHPPES INSTITUTE 



DATE 

3/10 
3/17 
3/21 
2/13 



DAYS, 



DC 



I 2 I 

I 2 I 

I ) I 

I 2 I 



STftLLCOP 
STflLLCOP 
STflLLCOP 
TtiOH^ 



22-12 UNIVERSITY DF COLORflDO* 

22-09 WORIflL HOSP SCH OF RflD-TECH 

22-08 COLORflOO CQLLEQE 

22-10 PIKES few COWUNITY COLLEGE 

22-02 HEN MEXICO HIGHUNOS UNIVERSITY 

22-06 ICH ICXICO STATE UNIVERSITY 

22-07 UNIVERSITY OF TEIAS-EL PASO 

22-03 PHOENIX INSTITIHE OF TEC»ta06Y 
SOUTH WUNTAIN CONMUNITY COLLEGE 

22-05 COCHISE COLLEBE 
19-11 OKLPHOm STATE UNIV-ABRIC/IW. SCI 



2/17 I 
2/19 I 
2/20 I 
2/2A 1 
2/27 I 
3/03 I 
3/06 I 
3/10 I 
3/12 I 
3/ IB I 
3/2A I 



2 I 

1 i 

2 I 



I 



3 
2 
2 
2 
2 
5 



THMPSON 
THOMPSON 
THCM>SON 
THOMSON 
TWDSON 
THON>SON 
THOIf>SON 
TWWSON 
THOWSQN 
THOMPSm 
THOMSON 



01-03 
01-04 
01-07 
01-12 
01-10 
01-15 
01-05 
01-06 
01-13 
01-09 
01-02 
01-01 



MCINTOSH COLLEGE INCt 
UNIVERSITY OF NEU HRI«>SHIRE I 
SMITH COLLEGE > 
WRVARO UNIVERSITY/RAOaiFFE COLLEGE I 
REGIS COLLEGE > 
COM. LEARNING CENTER SOMERVILLE, MASS I 



UNIVERSITY CF MASSACHUSETTS 
MCRST COLLEGE 
IWSSflCHUSEnS INST OF TECH 
SALEM STATE COLLEGE 
NOWICH UNI\CRSITY 
JMBOM STATE COLLEGE 



2/10 I 
2/12 I 
2/17 I 
2/19 I 
2/24 I 
2/26 I 
3/03 I 
3/06 I 
3/10 I 
3/12 I 
3/17 I 
3/20 I 



2 

3 

2 

3 

1 

2 

3 

2 

2 

3 

2 

2 



TIMESON 

TUVESON 

TUVESOH 

TUVE;W 

TIMSUN 

TUVESON 

TUVESON 

TUVESON 

TUVESON 

TUVESON 

TUVESON 

TUVESON 



1 I 
I 
I 
I 



CENTM. BRPTIST COLLEGE I 2/10 I 2 ' 

UNIVERSITY OF CENTRAL ARKANSAS I 2/12 1 3 1 

HID PLAINS COWUNITY COLLEGE I 2/17 I 2 I 

CWDRON STATE COLLEGE ' 2/20 I 2 1 

NATIONAL COLLEGE ' 2/2* ' 3 I 

SOUTH D«<OTft SCH OF MINES » TECH I 2/27 I 2 1 

COLOmOO MOUNTAIN COLLEGE I 3/03 I 

UNIVERSITY OF COLUfiflW I 3/04 I 4 

UNIVERSITY OF EVANSVILLEt I 3/10 I 2 

STATE AREA VOC-TECH SCH-JACKSON I 3/13 I 2 

UAYNE STATE COLLEGE ' 3/18 I 2 I 

INOIMM UNIVERSITY BLOOMINGTON I 3/24 I 2 I 

3 I 

2 I 

3 I 
2 I 
2 I 
2 I 

1 I 

2 I 

2 I 

3 I 



14-01 
14-02 
21-06 
21-03 
21-02 
21-01 
21-07 
21-08 
13-10 
13-06 
21-05 
12-04 



UALLMAW 

MLSMMM 

UALOMANN 

UALDMANN 

UALDMANN 

yALOMANN 

UALDMMM 

UALONAW 

UALDMANN 

UALOMMM 

UALOMANN 

UALOMAm 

UALDMAW 



16-10 
16-12 
16-09 
16-08 
16-11 
16-03 
16-05 
16-02 
16-01 
16-13 
13-08 



UEBSTER UNIVERSITY I 2/10 I 

UNIVERSITV OF MISSOURI-ST LOUIS I 2/12 I 

COLUMBIA COLLEGER ' 2/17 I 

UNIVERSITY OF MJ.SSOURI-COLUMBIfl I 2/19 I 

MISSOURI SCH DO:TORS ASST I TECH I 2/24 I 

EUREKA COLLEGE ' 2/27 I 

SPOON RIVER COLLEGE ' 3/03 I 

ST FW»CIS HJSP SCH OF NURSING I 3/05 I 

Wmm COUIGE ' 3/06 I 

ICTRO BUSINESS COLLEGE ■ 3/10 I 
ST AREA VOCATIONM. TEQWCAL SCHOOL I 3/12 I 



UATSON 

UATSON 

UATSON 

UATSON 

UATSON 

UATSON 

UATSON 

UATSON 

UATSON 

UATSON 

UATSON 
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ID* SCHOOL NflIC 



16-14 SOimCAST HISSOURI STATE UNIV 

U-13 ALCORN STATE UNIVERSITY 

H-11 TOUGflLOO COIEBE 

14-10 DELTA STATE UNIVERSITY 



DATE 

I 3/17 I 

I 3/24 I 

I 3/31 I 

I 4/02 I 



• 

DAYS 
•I 
4 I 

2 I 
2 I 
2 I 



OC 

MTSON 
UATSGN 
UATSON 
UATSON 



CCJWENTS 
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January 16, 1986 



Dear Financial Aid Administrator: 



A few days ago one of our representatives called you to arrange an 

interview for the Title IV Quality Control Project. I would like to 

confirm that the date for this interview is . A 

project interviewer will call you a few days before this date to 
reconfirm the visit. 

The study is being conducted in accordance with section 552a (e) (3) 
of the- Privacy Act of 1974, 5 U.S.C. 552a (e) (3), and section 5b. 4 of 
the Department of Education regulations implementing that sect ion , 34 CFR 
3b. 4. Access to your student records is authorized under Department of 
Education regulations implementing the Title IV programs, 34 CFR 
668.12(c)(3). 

If you have any questions or you cannot keep this appointment, please 
call Beth Schwartz, of Advanced Technology, collect at (703) 620-8253. 

I appreciate your cooperation with this important study. 



Sincerely, 



ADVANCED TECHNOLOGY, INC., . 

yCSarol M. Miller 
/ Pre 



Project Manager 
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Items Needed At Time of Visit 

0 A list of recipients of Pell Grants 

« An unduplicated list of students who have been awarded and have 
accepted Campus-Based aid (a list on which the names of recipients 
appear only once, even if they have received aid from more than 
one Campus -Based program) 



• 



A list of students whom you have certified for GSL's 

• All procedures for determing cost of attendance {including budgets) 

• All policies on whom to validate/verify 

• All packaging policies 

• Special forms for documenting need adjustments 

• Refund/ repayment policies 

• Any forms you use in your quality control procedures 

The data collector will also need to review the 1984-85 enrollment status 
of 2 preselected Pell Grant recipients. Please advise the Registrar's 
office that the data collector will need access to the 1984-8S records. 
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Ccntacr !^ame 



951821 



INST- CODE CALLER 

INST. JiME ^I-M£ 

DATE 



City State 

RESCHEDULE YES 

NO 



TELEPHONE NO. 



TITLE IV STUDY SCHEDULE COMriKKATZOM 



Good (moming/afttmoon) (Ms. /Mr.) • I ' 

froo Wtstat, Inc., in RocJcvillt, Maryland. A coupla of wttks ago Mr. 
Ernst Btckar from tht Dtpartmtnt of Education sant you a Ittttr 
dtscribing tht Titlt IV Quality Control Study that Advanctd Tachaology 
and Wtstat will conduct and tht kinds of activitits wt will bt ptrforaing 
at 300 institutions throughout tht U.S. 



1. Havt you rtctivtd tht Ittttr and had a chanct to rtad it? 

YES (GO TO 3) 

NO 

CAN^T RECALL 

WANT MORE INFORMATION 

2. Advanctd Ttchnology, Inc., and Wtstat art undtr contract to tht 
Dtpartmtnt of Education to conduct a Quality Conti'ol Study for tht 
Titlt IV studtnt aid programs. Tht ma:or objtctivts of tht study art 

to: 

0 Dtttrmint paymtnt and award error ratts for thost programs by 
inttrvitwing parents « studtnts, and institutions 

• Ba£int tht probablt causts of thtst trrors 

• Dtvtlop corrtctive action proposals to rtduct paymtnt error 

Tht institutional phast of this ytar's data collection is dtsigntd to 
visit tach of tht samplt institutions, inttrvitw tht financial aid 
administrator, and compilt data on a samplt of Ptll and Crmpus-Bastd 
rtcipitnts and GSL ctrtif ications. Wt will bt making thtst visits 
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bttvran Ftbriusy 10 *nd March 21. Th« inttrvitw will t»ke about »n hour 
and a half. W| will naad to salact a aamplt of your axd rtcipitnts from 
your'c«cofdi and coapila information on thoia atudanta. Wt tstitnatt that 
tha a^;*raga viait will taka ona to tvo dayt. but you will nttd to be 
availabla -only for tha intarjfiaw. Othar aspacts of data coUtction do 
not raguira your prascnca. 



3. Doas your institution hava branch canpuaaa? 

T£S 

NO (GO to 5) 



4 I will naad a list of branch caiipusaa and tha unduplicatad count of 
Titla IV raclpianta for 1985-«6 at aach. Could you giva thaa to mt? 

(IF THE RESPONDDIT CANNOT GIVE UNDUFLICAIED MCIPIDITS, ASK FOR 
APPROXIMATE ENROLLMENT AT EACH BRAilCB CAMPUS) 

CAMPUS UNDUPLXCATZD WBCIFIENTS /ENROLLMENT: 

PELL CAMPUS BASES GSL 



1 

2 

3 . 

4 — 

■ 5 - 

6 . 

7 

(I? MORE IHAS 7 BRANCHES, CONTXNUS ON BACK OP THIS PAGE) 
(GO TO 8) 

5. .What is tha nuaiJar of Pall Grant racipiants for 1985-86? 
NUMBER 

DON'T KMOM ~ - ^ ^ 



6. «hat is tha astiaatad nuabar of unduplicatad Caapua-Basad rtcipxants 
for 1985-86? 

NUMBER 

DON'T KNOW 
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7. What if tht tttimattd nuiobcr of GSL ctrtif ications for 1985*86? 

2 IIUMBER 

• DON*T KNCW 



A. For planning purpostt^ wt havt tstablishad a tantativa sehadula 'to 
visit all institutions this spring and trust that oost institutions 
will try to aceocBmodata that sehadula so tha Oapartisant of Education 
will racaiva our findings by mid*sumar. 

I havf a ehaeklist of ittms to ask you ragarding our visit to ^*7ur 
institution: 

9. Our intarviattr is tantativaly schadulad to bagin tha visit to your 

institution on _at 8:00 a.0. Is that data and tima 

aeeaptabla to you? 

YES (GO TO 12) 

NO, DATE UNACCEPTABLE {GO TO 10) 

8:00 A.M. UNACCEPTABLE. What tima can our visitor arrive 

to gat in a full day's %iork? 



RECORD TIME (GO TO 12) 



10. Bacaiisa our intarviawars hava so much travailing to do, it is 
ia^ortant that va ba abla to stop at all tha sanpla institutions in 
ona city or araa on a singla visit. Panding eonfinaation frooi othir 
institutions in your araa, or 

vould ba good altamata start datas for us. Would thay ba aeeaptabla 

to you? 

YES (00 TO 12) 

MO 



11. raayAay ^ that wa hava to kaap a vary tight sehadula, what 
would ba tha elosast aeeaptabla data to tha data I originally 
suggastad? I originally suggastad (RECORD ALXERIATE 

WSt) ^ . 



12. Would any local holidays, school vacation pariods, or othar avants 
intarfara with (our proposad visit/any of tha altamata datas)? 
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NO (GO TO 16) 



13. Ii that a local holiday which might affact othar institution* in your 
araa, or is it spacific to your institution?^ 

HOLIDAY. MIGHT AFFECT OTHER INSTITUTIONS 

SPECIFIC TO OWN INSTITUTION 



14 Baeausa our intarviawara hava ao mich travailing to do, it i« 
i«Dortant that wa ba abla to atop at all tha saapla institution* in 
onTcity or araa on a singla visit. Panding confinsatioo ttm othar 

institutions in your araa. — ■ — ■ ■■ ' 

would ba good altamata start datas for us. Mould thay ba accaptabla 

to you? 

______ YES (GO TO 16) 

NO 



15. Kaaping in mind that wa hava to Jcaap a vary tight schadula. >#hat 
would ba tha closast accaptabla data to th. <i*2^ SSSSii? 
nggastad? I orignally suggastad <RBtt»0 ALTERNATE 



DATE) 



16. Our intarviawac will naad to raviaw individual studant ^oMncxkl aid 
racords. Ara tha racords for all studants who ara racaiving Titla IV 
aid kapt in tha Studant Aid Offica? 



YES 
NO 



(IP NO. ASK WHERE THET ARE LOCATED AND RECORD. IF 
LOfiAIED AT OTHER THAN CAMPOS. OBTAIN NAME AND ADDRESS 
OP LOCAIIQM AND NAME AND PHONE NUMBER OP INDIVIOUAL 
RESPCNSIBIX FOR MAINTAINING THESE RBCC»DS. ) 



17 Whara is tha Studant Aid Offica locatad? (RECORD BUILDING. STREET 
ADDRESS, IP APPROPRIATE. FLOOR AND ROOM NUMBER.) , 
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18. To gtt All -tht information thty nttd, ouc inttrvitwtri ofttn htvt to 
visit tht iJigistrar and Bursar, as wall as tht studtnt aid of£ict. 
Art both tht Rtgistrar and BU4.sar locattd in tht samt building as 
your offict? 

YES (GO TO 21) 

NO 



19. Whtrt IS tht Rtgistrars office? 
(RECORD ANSWER) 



20. Whtrt IS tht ^Bursar's offict? 
(RECORD ANSWER) 



21. Our data colltctor will ^havt a rtcipient/applieant stapling proctdurt 
to follow upon arrival. Kowtvtr^ tht sasplt stltction will b« 
accompli shtd much fasttr if thtrt is a cdKbintd list of all 1985H|)S 
SEOG. Colltgt Hork--Study, and NDSL, rtcipitnts at your institution, a 
list of all Ptll Grant rtcipitnts for 1985-86, and a list of GSL 
ctrtifications madt for 1985-*86. 



22. Do you now havt such lists? 

YES 

NO (GO TO 24) 



23. Art all Campus -Bastd rtcipitnts on ont list, or do you havt a 
stparatt list for tach program? 

^ ALL ON ONE LIST (GO TO 25) 



-SEPARATE LISTS {GO TO 31) 



OMt -^aka colltctor will nttd instructions upon arrival on how to ttll 
from which programs tach studtnt on tht list is rtctiving aid. 



24. Could you compilt such a list? 

YES Wt would vary much apprtciatt your doing that btfort 

tht datt of our visit. 
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25. -How dr« r«cords physically stored (fil«s in cabintt, cards m 
filabox)? RECORD ANSWER 



Our data collactor will hava to usa tha fllas upon arrival and will 
naad somaona to axplain your filing or racord-kaaping systam. (60 

TO 26) 



26. What is tha last day of classas in this acadamic yaar/tam? 



27. Is thara any particular naarby hotal or fflotal which you would 
racomand tQ a parant or ttudant f roai out of town who was visiting 
your school? (DC NOT PROBE FOR ANSWER TO THIS QUESTION 



' 28. Thank you again for your cooparation. W» will sand yorx 
confirmation of tha visit in tha mail* 



CO^IMENTS 
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SPRING, 1986 DATA COLLECTION 



INTERVIEWER VALIDATION REPORT 

Interviewer 

Institution Visited/ID 

Financial Aid Administrator 

Telephone Number 

Date of Visit 

Validation Calls (Enter call--back time in next column) 

Date: 

Time: 



Hello, this is from Advanced Technology, Inc. I am 

calling about the visit of to your institution to 

collect .iata for the Quality Control study we are doing for the 
Department of Education. In order to assure the quality of this data, we 
are calling the participating institutions to evaluate the performance of 
our field personnel. 

^ ' was scheduled to interview 

Date of visit Name of Interviewer 

you and collect some data from some of your student files. 

1. Did (he/she) arrive on time? 

1. Yes 

2. No 

2. Did (he/she) present (his/her) credentials? 

1. Yes 

2. No 

3. Did (he/she) conduct the interview with you in a professional manner? 

1. Yes 

2. No What in particular did you find unprofessional about 

(his/her) conduct? 
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NAME: 

REGION: 

DATE: 



DISCUSSIOif TOPICS: 



MATERIALS: 



INTERVIEWS: 



CASH ADVANCE: 



TRAVEL ARRANGEMENT: 



EXPENSE REPORTING: 



OTHER: 
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Txar A r^r\i t c/^riM DOnRT PUQ TCk HKCUS^ WITH 

FIELD DATA COLLECTORS 

DATA COLLECTOR: 

DISTRICT: 

PERSON FILLING OUT THIS FORM: 

PROBLEM REVEALED BY: 

( ) Editing 

( ) Verification Call 

( ) Field Observation 

( ) Other (Specify) 

DISCUSS WITH DATA COLLECTOR: 

( ) Inrjmediately ( ) In Scheduled Call 

DESCRIPTION OF PROBLEM: 
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UNITED STATES DEPARTMENT OF EDUCATION 
OFFICE OF THE ASSISTANT SECRETARY FOR POSTSECONDARY EDUCATION 



Dear Student t 



According to our records, yow are receiving financial aid from at least one of the following federal 
programs: the National Direct Student Loan (NDSL), Supplemental Educational Opportunity Grant (5E0G), 
College Work-Study (CW-S), Pell Grant, or Guaranteed Student Loan prograii. The Office of Poataecondary 
Education of the Oepartsient of Education is conducting a quality control study to determine the aiwunt 
and type of payment errors being made in administering these programs, snd ths probsble causes of 
errors. The study will enable th^ E)epartirtent of Education to take corrective sctiona to allsviate any 
major errors identified. 

You and your parents have been randomly selected to participate in the study, and m need epecific 
information from you for the study to be successful. A represent stive from Westat, Inc., a research 
firm located neer Washington, D.C., will be contacting you in a few weeks to arrange an intf>rview. 
You will be requested to show certain financial records to verify the information submitted on your 
19e5*86 financial aid application. We appreciate your cooperation, since the results of this study 
will be used to improve the delivery of student financial assistsnce. 

To keep the interview short and effective, please follow the instructions on the enclosed form, 
"List of Documents Necessary for this Study." In most cases, these instructions simply request 
thst you obtsin certain documents to verify your income and assets. The Westat interviewer will 
ssk you to show these documents during the interview. 

If you filed a 1984 Federal Income Tax Form, we request thst you voluntarily sign the enclosed lf\cme 
Tax Form Release Statement, and return it to Westat as soon ss oossible in the enclosed postage-paid 
envelope. Westat will send the release statement to your IRS Service Center, and arrange for s copy 
of your tax form to b© sent directly to Westat. While it is not mandatory that you sign this release, 
this procedure is necesssry to guarantee that we obtain a complete picture of the type of errors msde 
in connection with student financisl assistsnce programs. 

I want to emphasize that the help you and other atudents give us is vitel to the success of the study. 
Because only a acnall representative sample of students will be interviewed, your information will play 
an important irt in the study's findings. Future changes to Federal student financial aid programs 
will rely, ir rt, on the outcome of this atudy. 

This study is being conducted according to the regulations of the Privacy Act. The interviewer has 
signed a confidentiality statsment under which he or she has eworh not to reveal to anyone not con- 
nected with this study sny information you provide. If you hcve any questions sbout the study, or any 
of the things you hsve been asked to do, please call the Westat represent stive at (800) 544-7755 toll 
free, and mention Title IV. 

Thank you in advence for your assistance and cooperation. 



Sincerely, 




Ernst G. Becker, Director 
Diviaion of Quality Aaaurance 
Debt Collection and Management 



Aaaiatance Service 



Enclosures 




4UU MARYLAND AVE. SW WASHINGTON. D C 20202 




ANSWERS TO QUESTIONS THAT ARE OFTEN ASK^.D 



WHO IS CONDUCTING THE STUDY? 

This Title IV Quality Control Study is being conducted by the Quality Assurance Division of 
the U.S. Department of Education. Westat, Inc., a social science research firm located 
near Washington, D.C., is uider contract to the Department of Education to conduct the 
survey interviews. 

WHY IS THE STUDY BEING CONDUCTED? 

The information is being collected to determine the types o'f errors being made on financial 
aid applications, why they are being made, and to rcccmmen^J appropriate changes. 

WHY WAS I CHOSEN? 

A sample of students and their parcnta were randomly selected from lists of those students 
receiving either a Pell Grant, a Supplemental Educational Opportinity Grant (SCOG), a 
National Direct Student Loan (NDSL), a Guaranteed Student Loan (GSL), or College Work-Study 
(CW-S) assistance inder Title IV of the Higher Education Act. Your name was one of those 
chosen. 

AM I REQUIRED TO PARTICIPATE? 

Participation in this study is volintary, unless you sre receiving a Pell Grant. However, 
we would spprcciate your help in doing this survey. The help that you and other students 
and parents give us is vital to the success of the survey. The authority for collecting 
this informstion is in Title IV of the Higher Education Act of 1965 [Section 411(b)(2) ss 
amended by 20 U.S.C. 10708(b)(2) and 45 CFR 190.12]. 

HOW WILL I RECOGNIZE THE INTERVIEWER? 

Prior to making personal contact at your household, the Westat interviewer will attempt to 
reach you by telephone (if a number can be obtained). At that time he or she will answer 
any questions, set up a time for the interview, and tell you about tt>e identification he 
or she will be carrying. All Weatat interviewers carry identification cards with their 
pictures. 

WILL THIS INFORMATION BE KEPT CONFIDENTIAL? 

This study is being conducted according to the regulations of the Privacy Act. The inter- 
viewer has signed a statement swearing not to reveal any information obtained during this 
interview, except for the purposes of this study and as required by lew. 

HOW LONG WILL THIS INTERVIEW TAKE? 

Our experience has shown thst most interviews take approximstel y twenty to thirty minutes. 
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LIST OF OOCUtfNTS NECESSARY FOR THIS STUDY 

hM f»tt of this itudy, t»» »to«t«t int.rvitwr tdll Mk you to •»»»» hia ot h»r v«rioi» doco- 
■antt and fonts «hich v«rify tte infonMtion r«port«d on th« tpplicttion for TiU« IV Msistance. 
Th* following chart lists tha rmma of thaae docuaants and fonia. If they apply to you, plains 
hdve thea ready to shoN the interviewer. 



IF YOU REPORTED ON THE 
APPLICATION THAT YOU: 


BE READY TO SHOH THE INTERVIEWER: 


la Are a UaS* 
citizan 


A docuwnt auch aa a birth certificete or paaaport that 
vea^ifiea your citizenship. 


2a Are tn eligible 
norv-citizena 


A docu«ent auch aa a paaaport, Fora U151, or Fora 1-94, that 
indicataa you are an eligible non-citizena 


3a Are married , 

aeparated or divorced* 


A docunsnt auch aa a Mrriaga certific^ata, divorce decree, legal 
aaparetion agresMnt or a(<9ropriete oocuaent to verify Mritai acacuaa 


4a Did not live with your 
psrente for iwre than 
aix ««eeks during 1984a 


Docuwenta, auch aa rent receipte, leeae agraawanta^ cancelled 
checks, or Mortgage atatmenta that indicate you did nsl : 
with your parcnta in 1984a 


5a Did no^ live with your 
peranta for Mre than 
aix weeks during 1983a 


Oocunenta, auch aa rent receipts, leaae egresMnta, cancelled 
checks, or aarlgsge stetSMnts that indicate you did not live 
with your paranta in 1985a 


6a (Or your apouse) 
had incone in 1984a 


a If vou filed a 1984 Federal Tax Return: 


NOTE; If you are narried 
and filed aeparately fron 
your apouae in 1984^ theae 
inetructiona alao apply 
to your apouaalao apply 
to your apouse a 


>*fiEFORE THE INTERVIEW: While it ia not mandatory, wt requeet that 
you voluntarily coaplate and aign tha ancloaed releaae atats«ent 
(Form 4506' eo that a copy of your 1984 Federal Incoae Tax Return 
«xi all aupporting achedulea can be aent to Waatata Refer to the 
special inetructiona for coaplating thia fonia Do not enplose a 
4^ check with thia fonia Tha fee will be peid by Weatata Pleaae 

1 complete and return AS SOON AS POSSIBLEa 

DURING THE INTERVIEW: If y;>J have one on hand, you aho^ild be 
prepared to ahow a copy of your 1984 tax return or workaheet 
\.to tha interviswera 

a If vou had earned incoae but did not file e 1984 Federal Incosie 
Tex Return, have a W.2 or 1099 that indicatea the amount you 
earned in 1984. 

• If vou received non-texable incoae in 1984. auch aa Social 
Security benefita, welfare benefita, child aupport or Veterans' 
benefite (other than educational benefita), be ready to ahow the 
interviewer a ststeweni or fone froa ths sppropriate agency that 
indicates how isuch you received in 1984a 
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INFORMATION UPDATE SHEET 

PLEASE FILL OUT THIS FORM AND HAIL IT IN THE ENCLOSED ENVELOPE 



PLEASE 
COMPLETE 



Studant's Name 



Mother's Name 



Mother's 



First 



Last 



First 



Last 



Address Street # 



Street Name 



City 

Mother's Telephone # i 1. 

Area Code 



State 



^ Father's Name 



First 



Last 



Initial 



Initial 



Apt. # 



Zip 



Initial 



COMPLETE IF 
DIFFERENT 
THAN ABOVE 



Father ' s 

Address Street # 



City 

Father's Telephone # L 



Street Nassie 



State 



Area Code 



Apt. # 



Zip 



COMPLETE IF 
YOUR ADDRESS 
IS DIFFERENT 
THAN THE ONE 
TO WHICH THIS 
LETTER WAS 
HAILED 



PLEASE 
COMPLETE 



Student's ^ 

New Street # 

Address 



Street Naiie 



City 



State 



Student's Telephone # i L 



Area Code 



Apt. # 



Zip 



OFFICE USE ONLY 



304 



D-28 



(STUDENT) 



WE STAT 
Title IV Q.C. Study | 



ron. 4506 

(Rtv Octobtr 1985) 

Dtptrtmtnt of tht Trt asury 
inttrnal Rtv*nut Scfvice 



Request for Copy of Tax Form 
or Tax Account Information 

► P(#«MrtadlMtruct)ofitb«foracompl«tkif this form. 



OMBNo 1&4S0429 



Important: Full payment must accompany your request. 



1 N<rr.« of Uxpayer(s) as shown on Xtx form 



2 Current name and a<klress 



3 If information is to be mailed to someone else, show the third party's name and address 
WESTAT 

1650 Research Boulevard 

Rockville, Md. 20850 (Title IV Study) 



4 If name in third party's records differs from item 1 above, show here (See instructions 
for Items 3 and 4.) 

See Transmittal Document (attached) 



5 Social security or employer identification number as 
shown on tax form 



6 Spouse's social securrty number as shown on tax form 



7 Tax form number (Form ICMO, 1040A. etc.) 



1040, 1040A or 1040EZ 



$ Tax period(s) (1983, etc ) (No more than 4 per request) 



1984 



Amount due: 

• checked in item 10 ... . 
b Number of periods requested in 

item 8 

c Total cost (multiply item 9a by item 

9b) 

MMk0 ch0€k psysbh to IRS 



10 Describe what you want (Check only one box) 

$4 25/each period requested [Kl Copyof tax form and all attachments 

Note: If you netd thtse copies for court or admrnistrative proceedings, check here \ | also. 



$2.25/each period requested 



□ 



Tax account information only 









Telephone number of requester 


Please | 


• 


1 


(301) 251-1500 


Sign 


Signjture 


Date 


Convenient time for us to call 


Here | 






9:00 a.m. • 5:00 p.m. 
Monday • Friday 




Title (If Item I above ts a corporitKMi. partnership, estate or trust) 





Instructions 

Privacy Act and Paparwork Reduction Act 
Netlca. — We ask for ttits information to 
carry out the Internal Revenue laws of the 
United States. We ntwd the information to 
gam access to your return in our files and 
properly respond to your request. If y^'j do 
not furnish the infofmation. we may not be 
able to fill your request. 
Purpota of Form. — Use this form to 
request a copy of a tax return or tax account 
information. 

Note: If you htdyovr return filled oat by a 
psid preparer, check first to see if you can 
get a copy. This may i ave you tKth time and 
money. 

If you are not the taxpayer shown in item 
1, you must send a copy of your 
authorization to receive the information. 
This will generally be a power of attorney, 
tax information authorization, or evidence 
of entitlen>ent (for Title 1 1 Bankruptcy or 
Receivership Proceeding), tf the taxpayer is 
deceased, you must sand enough evidence 
O stablish that you are authorized to act 
R j^C ^he taxpayer's estate. 



Tax returns and return information about 
joint returns may be disclosed to either the 
husband or the wife. Only one signature iS 
required. If your name has changed, sign 
Form 4506 exactly as your name appeared 
on the return and also sign with your current 
name. 

Please allow at least 45 days for delivery 
when requesting a copy of a return, or at 
least 30 days when requesting return 
information. (You must allow at least 6 
weeks processing time after a return is filed 
before requesting a copy or other 
information.) 

Corporations, Partnerships, Estates, and 
Trusts.-<— For rules on who may obtain tax 
information on the entity, see Internal 
Revenue Code section 6103. 
Items 3 and 4. — If you have named 
someone else to receive the information 
(such as a CPA. scholarship board, or 
mortgage Issuer), you mutt Include the 
name of an Individual with the address in 
item 3. Also, be sure and write the name of 
the client, student, or applicant in item 4 If it 
is different from the name shown in item 1. 



Student applying for financial aid. Show the 
student's name in item 4 so the scholarship 
board will know what file to associate the 
return infornwtion with. If we cannot find a 
record of your return, we will ratify the third 
party directly that we cannot fill the 
request. 

Item 5.— For * dividuals. the social 
security number is written 00(^00•0000. 
For businesses and certain others, the 
employer identificatton number is written 
00-0000000. Please separate the nine 
digits as shown, to distinguish the type of 
number beii^ reported. 
Item 8.— Enter the year(s) of the tax form 
you are requesting. For fiscal-year filers or 
requests for quarterly returns, enter the 
date the period ended. If you need more 
than four different periods, use additional 
request forms, Returns which were filed six 
or more years ago may not be available for 
making copl^. However, tax account 
information is generally still available f or 
these periods. 

(Continued on t>ack) 
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UNITED STATES DEPARTMENT OF EDUCATION 
OFFICE OK THE ASSISTANT SECRETARY FOR POSTSECONDARY EDUCATION 



Dear Student, 



According to our records, you are receiving financial aid from at least one of the following Federal 
programs J the National Direct Student Loan (NDSL), Supplemental Educational Opportunity Grant (SEOG), 
College Work-Study (CW-S), Pell Grant, or Guaranteed Student Loan program. The Offic# of Postsecondary 
Education of the Department of Education is conducting a quality control study to determine the amount 
and type of payment errors being made in adminiatering these programs, and the probable causes of 
errors. The study will enable the Department of Education to take corrective actiof.s to alleviate any 
major errors identified* 

You and your parents have bcon randomly selected to participate in the study, and m need specific 
information fro« you for the study to be successful. A representative from Westat, Inc., a ref«^srch 
firm located near Washington, O.C., will be contacting you in a few waeks to arrange an interview. 
You will be requested to show certain financial recordii to verify the infurmation aiAMiitted on your 
1985-86 financial aid application. He appreciate your cooperation, since the results of this study 
will be used to improve the delivery of student finsncitfl assistanca. 

To keep the interview short and effective, please follow the instructions on the enclosed fore, 
"List of Documents Necessary for this Study." In most cases, these instructions simply request 
that you obtain certain documents to verify your income and assets. The Westat interviewer will 
ask you to show these documents during the interview. 

If you filed a 1984 Federal Income Tax Form, we request thst you voluntarily sign the enclosed Income 
Tax Form Release Statement, and return It to Westat as soon as possible in the enclosed postage-paid 
envelope. Westat will send the release statement to your IRS Service Center, and arrange for a copy 
of your tax form to be sent directly to Westat. While it is not mandatory that you sign this release, 
this procedure is necessary to guarantee that we obtain a complete picture of the type of errors made 
in connection with student financial assistance programs. 

I want to emphasize that the help you and other students give us is vital to the success of the study. 
Because only a small representative sample of studenta will be intervieweo, your information will play 
an important part in the atudy's findings. Future changes to Federal student financial aid programs 
will rely, in part, on the outcome of this study. 

Thia study ia being conducted according to the regulations of the Privacy Act. The interviewer has 
signed a confidentiality statement under Miich ne or ahe has sworn not to reveal to anyone not con- 
nected with thid study any information you provide. If you have any questions about ti.^ atudy, or any 
of the thinga you have been aaked to do, please call the Weatat repreaentative at (800) 544-7755 toll 
free, and mention Title IV. 

Thank you in advance for your assistance and cooperation. 



Sincerely, 




Ernat G« Becker, Director 
Division of Quality Assurance 
Debt Collection and Management 



Assistance Service 



Enclosures 




400 MARYLAND AVE. SW WASHINGTON. D C 202OJ 



D-3D 




n^rg%wr9r\r» m/-v ^TTfOTTi^KTC THAT 11 PF ^FTFM AR'J^^n 



WHO IS CONDUCTING THE STUDY? 

This Title IV Quality Control Study is being conducted by the Quality Assurance Division of 
the U.S. Department of Education. Weatat, Inc., a social science rcsesrch firm located 
near Washington, D.C. , is uider contract to the Department of Education to conduct the 
survey intervieMS. 

WHY IS THE STUDY BEING CONDUCTED? 

The information is being collected to deteiwine the types of errors being made on financisl 
aio applications, why they are being made, and to recommend appropriate changes. 

WHY WAS I CHOSEN? 

A sample of ttudenta ar^i their parents were randomly selected from lists of those students 
receiving either « ?«11 Grant, a Sipplemental Educational Opportunity Grant (SEOG), a 
National Direct Student Loan (NDSL), a Guaranteed Sti;dent Loan (6SL), or College Work-Study 
(CW-S) assistance under Title IV of the Higher Edur;ation Act. Your name Mas one of those 
chosen. 

m I REQUIRED TO PARTICIPATE? 

Participation in this study is voluntary, unless you are receiving a Pell Grant. Hov»ever, 
m wuld appreciate your help in doing this survey. The help that you and other students 
and parents give us is vital to the success of the survey. The authority for collecting 
this information is in Title IV of the Higher Education Act of 1965 [Section A11(b)(2) aa 
enended by 20 U.S.C. 1070a(b)(2) and 45 CFR 190.12]. 

HOW WILL I RECOGNIZE THE INTERVIEWER? 

Prior to making personal contact at your household, the Weatat interviewer will attempt to 
reach you by telephone (if a number can be obtained). At that time he or ahe will anawer 
any questions, set s time for the interview, and tell you about the identification he 
or she will be carrying. All Westat interviewers carry identification cards with their 
picturea. 

WILL THIS INFORMATION BE KEPT CONFIDENTIAL? 

This study is being conducted according to the regulations of the Privacy Act. The inter- 
viewer hsf* signed a statement awearing not to reveal any information obtained during this 
interview, except for the purposes of this study snd aa required by law. 

HOW LONG WILL THIS INTERV'^W TAKE? 

Our experience bBs ahown thst most interviews take approximately twenty to thirty minutes. 



LIST or DOCUICNTS NECESSARY FOR THIS STUDY 

Aa ptrt of thit study, ths Wt«t«t intirvitntr «iU ••*( you to ihow him or htr vtriout <k)cu- 
iwnta wxJ fonii *<hich vtrify tht infomttioo ptporttd on th« tppiicttion for Titit IV utittanct* 
Tht foUowing ch.rt littt tht nMtt of thttt docmwnti and fotcm. If thty apply to you, pltatt 
htvt th«i r««dy to show tht inttrvitwtr. 



IF YOU REPORTCD ON THE 
APPLICATION THAT YOUt 


<w ATAnv TM CitflM TUT tUTraWTTMPRf 
K Kc.AUY lu bnllW lr& irvitnTiCHLnS 


1* Are • U.S. citiztn 


A docunent such aa a birth certificate or paaaport that verifies 
your citizenship. 


2. Art «n tUgiblt 

non->citiZtn. 


A docuwtnt euch aa a paaaport » rons i-i>i, or hor» 1"t«» 
indicatea you are an eligible norv>cltizen. 


>. Are Mrricd, 

aeptrtted or divorced 


A dociMtnt such aa a aarriage certificate, divorce decree* legal 
separation agreement or appropriate docmwnt to verify marital atatua. 


4. Old not live with ya ^ 
parents Tor aore than 
aix wtkt during 
1984 and 198^* 


Oocumenta, auch aa rent receiptee leaae agree^nta, cancelled checks, 
or mortgage atetementa, that indicate you did not live with your 
□arenta in 1984 and 198^* 


3. Mere not listed aa an 
•xt«ptioo on your 
parenta' Federal Incoae 
Tex Return during 1984. 


A mpy nf th* front oaae of vour parenta* 1V84 Federal Income fax 
Return* 


6. Filed a 1964 

Federal Tax Return. 

NOTE! If you are «arriec 
and filed separately 
froA you apouae in 1984, ^ 
these inatructiona alao 
apply to /our apouae. 

V 


BEFORE THE INTEKVIEM: 


• Mhile It la not mandatory, we requeat that you voluntarily complete 
vxl aign the enclosed raleaae atatament uorm «?oo; ao tnai a copy o' 
your 1984 Federal Income Tax Return and all aupporting achedules csn 
be sent to Westst* Refer to the special inatructiona for completing 
thia form. Do not encioae a check with thia form. The fee will be 
paid by Meatat. Pleaae complete end return AS SOON AS POSSIBLE. 

DURING THE INTERVIEW: 


If you have one on hand, you ahouid be prepared to show a copy of 
^ your 1984 tax return or workaheet to the interviewer. 


7. Or your apouee) 
received Social 
Security benefits 
in 1984. 


A form or atatement from the Social Security office indicating 
the «M3unt of your (and your spouee*s) benefits in 1984. 


8. (Or your apouae) 
received non-taxable 

inCOHiV 111 

aa child support* 
general aaaiatance (for 
«xa«plet Aid to Depen- 
dent Children or other 
fonia of Mel fare)* or 
Veterw* benefits 
(other than educe- 
tional benefits)* 


a If you received child suppori; a document, auch as a court order 
or aeparation agreement, that indicatea the mwunt you (or your 
apouae) rcreivad in 1984. 

a If you received Aid to Dependent Children or other forms of 
welfares any documenta, auch aa a public assistsnce letter, 
thet indicate the mwunt of your (and yoiir apouse(s) benefits 
in 1984. 

a If you received Veteraoa* benefita (other than educational 
benefita)! a atatement or fona from the VA office i^ich 
indicatea the amount you (and your apouae) received in 1984. 
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INFORMATION UPDATE SHEET 

PLEASE FILL OUT THIS FORM AND MAIL IT IN THE ENCLOSED ENVELOPE 



PLEASE 
COMPLETE 



Student's Name 



Mother's Name 



First 



First 



Mother's 

Address Street # 



City 

Mother's Telephone # jL 



Last 



Last 



Street Name 



State 



Area Code 



Father's Name 



First 



Last 



Initial 



Initial 



Apt. # 



Zip 



Initial 



COMPLETE IF 
DIFFERENT 
THAN ABOVE 



Father ' s 

Address Street # 



City 

Father's Telephone # L. 



Street Name 



State 



Area Code 



Apt. # 



Zip 



COMPLETE IF 
YOUR ADDRESS 
IS DIFFERENT 
THAN THE ONE 
TO WHICH THIS 
LETTER WAS 
MAILED 



Student's 

New Street # 

Address 



Street Name 



Apt. # 



City 



State 



Zip 



PLEASE 
COMPLETE 



Student's Telephone # L. 



± 



Area Code 
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WESTAT 

Title IV Q.C. 



S*ndy| 



ro. 4506 

(Rev Q€tQ^ 1985' 

Dtp«itm«nt of tht TrtMufy 
Nittroa) RtvtflM Sffvict 



Request for Copy of Tax Form 

► P(«aMrMdlnftrMCtlorabtfor«cpmRMikif this^^ 





0MB No 1545-0429 




Experts 3-31'88 



important: Full payment must accompany your request. 



1 Name of taxpaytr(s)«s shown on U4 form 



2 Current name and address 



3 If information is to be mailed to someone else, show the third party's name and address 
WESTAT 

1650 Research Boulevard 

Rockville, Md, 20850 (Title IV Study) 



4 If name in third party's recofdi differs from item 1 above, show here (See instructions 
for items 3 and 4.) 

See Transmittal Document (attached) 



5 Social security. or employer identification number as 
shown on tax form 



6 Spouse's social security number as shown on tax form 



7 Tax form number (Form 1040. 1040A. etc.) 



1040, 1040A or 1040E2 

( Tax period(s) (1M3. etc.) (No more than 4 per request) 



1984 



Amount due: 

• CostcheckedlnitemlO . . . . 
b Number of periods requested in 

items 

t Tout cost (muttipty item 9e by item 

m 



10 DeKnbe what you want (ChecK only one box) 

$4.25/each penod requested [E] Copy of tax form and all attachments 

HoX%: If you netd these copies for court or admintstritiv9 prcct^ings. chtck h4r9 LJ a/w. 

$2.25/each period requested D Tax account information only 









Telephone number of requester 


Pleas* 1 


1 


1 

i 


(301) 251-1500 


Sign 


Signitufe 


Date 


Corwenient time for us to call 


Htr* 1 






9:00 a,m. - 5:00 p.m. 
Monday - Friday 




Title (If Item 1 above t% e corporation, partnenhtp. estate or trust) 





Instructions 

Privacy Act an4 Paparworlt Reduction Act 
Notict.— We ask for this information to 
carry out the Internal Revenue lews of the 
United States. We newd the information to 
gam access to your return in our files and 
properly respond to your request. If you do 
not furnish the information, we may not be 
able to fill your request. 
Purpose of Forfii.~Use this form to 
request a copy of a tax return or t£x account 
information. 

Note: If you had your return filled out by a 
p0id preparer, check first to see if you can 
get a copy> This may save you tx>th time and 
money. 

If you are not the taxpayer shown in item 
1, you must send a copy of your 
authorixation to receive the information. 
This will generally be a power ^' ^iiorney . 
tax information authorization, jr ovioence 
of entitlenf)ent (for Title 1 1 Bankruptcy or 
Receivership Proceeding). If the taxpayer is 
.^'^Msed, you must send enough evkience 
r> establish that you are authorized to act 
: ly^L the taxpayer's estate. 



Tax returns and return mfornriatkm about 
joint returns may be disclosed to either the 
husband or the wife. Only one signature Is 
required. If your name has changed, s>gn 
Form 4506 exactly as your name appeared 
on the return and also sign with your current 
name. 

Please altow at least 45 days for delivery 
when requesting a copy of a return, or at 
least 30 days when requesting return 
information. (You must allow at least 6 
weeks processing time after a return is filed 
before requesting a copy or other 
informatk)n ) 

Corporations, Partnerships, Estates, and 
Trusts. — For rules on who may obtain tax 
informatk)n on the entity, see Internal 
Revenue Code sectk)n 6103. 
Items 3 and 4.— If you have named 
someone else to receive the information 
(such as a CPA, KholarshIp board, or 
mortgage iuue*^ you mutt Include the 
name of ad In dual with the address in 
item 3 Also, be ure and write the name of 
the client, studtut, or applicant in item 4 if it 
is different from the name shown in item 1. 



Student applying for financial aid. Show the 
student's name in item 4 so the scholarship 
board will Know what file to associate the 
return information with. If we cannot find a 
record of your return, we will notify the third 
party directly that we cannot fill the 
request. 

tttm 5.— For individuals, the social 
security number is written 000^0000. 
For businessM and certain others, the 
employer Identification number is wntten 
OO^OOOOOOO* Please separate the nine 
digits as shown, to distinguish the type of 
number being reported. 
Item Enter the year(s) of the tax form 
you are requesting. For fiscal-year filers or 
request for quarterly returns, enter the 
date the period ended. If you need more 
than four different periods, use additional 
request forms. Returns which were filed six 
or nrK>re yean eto may not be available for 
making copies. However, tax account 
information is generally still available for 
these perkxis. 

(Continued on t^ck) 
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UNITED STATES DEPARTMENT OF EDUCATION 
OFriCEOFTHE ASSISTANT SECRETARY FOR POSTSECONDARY EDUCATION 



Otsr Partnti 




According to our rtcords, yout too or diughttr !• twtlving fininclil aid from .t Itaat ont of tha fol- 
l^-"g rSdaral pro,r««a. tha National Oir«:t St«d.nt Loan (NOSL). Supplam.nt.1 Educational Opportun ty 
Ct«nt (SEOG), CoUaga Work-Study (CH-S), Pall Grant, or Cutrantaad Studant Loan progri*. Tha Offica 
of Poataacondary Education of tha Department of Education ia conducting a quality control atudy to 
datar»lna tha aaount and typa of pay».nt arrora balng «ad« in adainlatarlng thaaa progra«a, and tha 
p«;ibla cauaaa of arror.. Tha atudy .ill anabl. tha Dapatt-nt of Education to taka corr«:tiva 
actions to allaviata any «ajor arrora idantiflad. 

You and your son or daughter h.v. ba«n randomly aalactad to particlpat. in tha -tudy. and naad 
.psciflc information frcn, /cu for i^. ^tuiy to to ...=.Hnfwl. A r«pr«.ntativ. f-o« ^'''^''t' ' 
raassrch firm located near Washington. O.C., -ill be cont«:ting you In a fa-j«k. to * J"^''' 

view. You will b. requested to .ho- certain financial r«:orda to verify th. 

your 1985-86 financial aid application. We appreciate your cooperation, elnee the result, of this 
etudy will be u.ed to improve the delivery of etudenc financial aaaletanee. 

To keep the interview ehort and effective, pleaee follow the inatructlons on the enelos* 

of Oocumente Neceaaery for thia Study." In moat cases, these instructions simply r^^ 

obtsin certsln documents to verify your Income and aaaets. Ths Westst interviewer will 
.how these docunents during the interview. ^ 

If you filed e 1984 Feder.l Incon« Tax Form, we requeet that you volunterily sign the enclosed Income. 
Tex Form R.leese St.tement. end return it to Weat.t s. soon .. possible In the ««lo.ed poeteg.-p.id 
envelope. Westst will send the relees. etetement to your IRS Service Center, and arrange for • copy 
Tr your tax form to be aant directly to Weetat. While l^ is not ««xl.tory that you algn thia rel»«e, 
Jhia procedure is necesssry to gusrantee that we obteln . complete picture of the type of errora made 
m connection with student finenciel e.elstwvs progrsms. 

I want to «ph.si« that the help you and other perenta give us is vital to the ^^J^* f ''mI: 
Beeauae only a ».all represent etive sample of students will be interviewed, your information will play 
TZortnll paTt in the .tudy'a findings. Future changes to F«ler.l student financial aid progr«s 
will rely, in part, on the o?jtc5:<je of this atudy* 

«^ . . - n • ^♦^^ .^-t.i^iUnns ?f '»7Uocr Act. The Interviewsr has 

II ed a Lnfidentl.lity s .tement ^der which he or eh. hes sworn not to oZy 
nected with this study sny informstion you provide. If you heve «.y ques Ion, *ou the »^ • " ""J 
of the things you have been ssUed to do. pl.soe call the Westst represent.tive at (800) 544-7755 toll 
fretf and mention Title IV. 

Thank you in advance for your assistance and cooperation. 

Sincerely^ 

Ernst C. Becker, Director 
Division of Quality Assurance 
Debt Collection and Management 
Asaiatance Service 

Enclosures 
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ANSWERS TO QUESTIONS THAT ARE OFTEN ASKED 



• WHO IS CONDUCTING THE STUDY? 

This Title IV Quality Control Study is being conducted by the (kjslity Assurance Division of 
the U.S. Department of Education. Westst, Inc., a social science research fii^ located 
near Washington, D.C., is inder contract to the Department of Education to conduct the 
survey interviews. 

a WHY IS THE STUDY BEING CONDUCTED? 

The information is being collected to detcimine the types of errors being made on financial 
aid applications, why they are being made, and to recomnend appropriate changes. 

a WHY WAS I CHOSEN? 

A ssnple of students and their parents were randomly selected from lists of those students 
receiving either a Pell Grant, a Supplemental Educational Opportunity Grant (SEOG), a 
National Direct Student Loan (NDSL), a Guaranteed Student Loan (GSL), or Collage Work-Study 
(CW-S) assistance inder Title IV of the Higher Education Act. Your name was one of those 
chosen. 

a m I REQUIi^ED TO PARTICIPATE? 

Participation in this study is voltntary, unless you are receiving a Pell Grant. Ho¥«ver, 
m wuld appreciate your help in doing this survey. The help that you and other atudents 
ano parents give us is vital to the auccess of the wrvey. The authority for collecting 
this information is in Title IV of the Higher Education Act of 1965 [Section A11(b)(2) as 
smended by 20 U.S.C. 1070a(b)(2) and 45 CFR 190.12]. 

a HOW WILL I RECOGNIZE THE INTERVIEWER? 

Prior to making personal contact at your household, the Westat interviewer will attempt to 
reach you by telephone (if a number can be obtained). At that time he or fhe will answer 
any questions, sst up a time for the interview, and tell you about the identification he 
or ahe will be carrying. All Wiestat interviewers carry identification cards with their 
pictures. 

a WILL THIS INFORMATION BE KEPT CONFir»:NT^AL? 

This study is being conducted according to the regulations of the Privacy Act. The inter- 
viewer has signed a ststement swearing not to reveal any information obtained during this 
interview, except for the purposes of this study snd as required by law. 

e HOW LONG WILL THIS INTERVIEW TAKE? 

Our experience has shown that most interviews take approximately twenty to thirty minutes. 
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LIST or DOCUMENTS NECESSARY FOR THIS STUDY 



PARENT 



As part of this study, the Westat interviiw^r will ask you to show him or her various docu- 
ments and forms which verif> the information reported oo the application for Title IV assistance. 
The following chart lists the names of these documents and forms. If they apply to you, pjease 
have them reedy to show the interviewer. 



IF YOU REPORTED ON THE 
APPLICATION THAT YOU; 


BE READY TO SHOW THE INTERVIEWER; 


1« Are marriiKi; 

separated, or divorced 


A document such as a marriage certificate, divorce decree, legal 
separation agreement or appropriate document to verify marital atatus. 


2. rileo a 193a 

Federal Tax Return 

NOTE; If you are married 
and filed separately from < 
your spouse in 1984, these 
instructions also apply 
to your spouse. 


"^BEFORE THE INTERVIEW: 


While it is not mendatory, we requeat that you voluntarily complete 
and aign the encloaed release statement (Form 4506) so* thst a copy of 
your 1984 Federal Income T^x Return and ell aupporting achedules cen 
be sent to Westat. Ritfer to the special instructions for completing 
this form. Do not enclose s check v^ith this form. The fee will be 
paid by Weatat. Pleaae complete and return AS SOON AS POSSIBLE. 

DURING THE INTERVIEW: 


If you have one on hand, you should be prepared to show a copy of 
^your 1984 tax return or worksheet to the interviewer. 


3. (Or your epouse) 
received Social 
Security benefits 
in 1984. 


A form or statement from the Social Security office indicating the 
amount of your (and your spouse's) benefits in 1964. 


4. (Or your dpouse) 
receivmd non*taxable 
incone in 1984, such 
as Child support, 
general assistance 
(for example. Aid to 
Dependent Children or 
other forms of 
•«lfare), or Veterans* 
benefits (other than 
educational benefits) 


e If you received child support: A document, such as s court 
oroer or ssparauxon agreement tn»t aiiuau«*»«i» «wvvk>^ 
(or your spouse) received in 1984. 

e If you received Aid to Dependent Children or other forms of 
welfare; eny documents, such as a public assistance letter, 
that indicate the amount of your (and your spouse's) benefits 
in 1964. 

f If you received Veterans' benefits (other than educational 
benefita): a statement or form from the VA office which 
indicetes the amount you (and your spouse) received in 1964. 


5. (Or your apouse) paid 
medical or denal 
expenses in 1964. 


A document, such as 1040-Schedule A, cancelled checks, or a statement 
from a hospital or doctor that indicates the amount you (and your 
spouse) paid in 1984. 
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INFORMATION UPDATE SHE£T 


1 
"1 

h 
I 

I 




PLEASE FILL OUT THIS FORM AND MAIL IT IN THE ENCLOSED ENVELOPE 


1 

&' 

i 






( Student' s Name 

First Last Initial 










/ 




MATT XIATV TP 

YOUR ADDRESS 


First Last Initial 


< 




IS DIFFERENT 








THAN THE ONE < 
TO WHICH THIS 


Fatner s Name 

First Last Initial 






LETTER WAS 








MAILED 


New 

Address btreet tf atreec name ^P^* ^ 








City State Zip 


1 
f 




COMPLETE IF 
YOU HAVE A NEW 


J Telephone # f > 
1 Area Code 




OR UNLISTED 


I 


5 
J 




NUMBER 










OFFICE USE ONLY 








(PARENT) 


s 

1 

-'I 


I o 
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WE STAT 
Title IV Q.C. Study 



f.cm 4506 


Request for Copy of Tax Form 


0MB No 1545-0429 


(Rtv.Octobtrl985) 


or Tax Account Information 


Expirts 3-31*88 


Otpaitmtflt of tht Trtasury 
Inttrntl RtvtflM S«rvi€t 


► PI«aMrMilmtrucll»fitktforec«iii|i^^ 



'>3 



Important: Full paymsnt must accompany your request. 



I Name of taxpaytr($} as shown on Ux form 


5 Social security or employer identification number as 
shown on tax form 


2 Current name and address 


$ Spouse's social security number as shown on tax form 


7 Tax form number (Form 1040, 1040A, etc.) 
1040, 1040A or 1040EZ 


3 If information is to be mailed to someone else* show the third party's name and address 
WESTAT 

1650 Research Boulevard 

Rockville, Md. 20850 (Title IV Study) 


• Tax period(s) (1983. etc.) (No more thar 
1984 1 1 


4 per request) 


9 Amount due: 

a Cost checked in item 10 .... 
b Number of periods requested in 

c Total cost (muKiply item 9a by item 
Afa*e chtck payabh to m$ 


$ 


4 If nanoe in third party's records differs from item I above, show here (See instructions 
for Items 3 and 4.) 

See Transmittal Document (attached) 





10 Describe what you want (Check only one box) 

$4.25/each period requested [E] Copy of tax form tnd all attachments 

Note; If you need these copies for court or admintstnttv proceedings, check here LI a/so. 



$2 25/each penod requested 



□ 



Tax account information only 









Telephone number of requester 


Please j 


• 


1 


(301) 251-1500 


Sign 


Signature 


Date 


Convenient time for us to call 


Here | 






9:00 a.m. - 5:00 p.m. 
Monday - Friday 




Title (H item 1 abcvt ts a corporation, partntfship. ttUtt or trust) 





Instructions 

Privacy Act and Paperwork Reduction Act 
Notict*— We ask for this information to 
carry out the Internal Revenue laws of the 
United States. We newd the information to 
pin Kcoss to y^r return in our files ard 
properly respond to your request. !f you do 
not furnish the infonnation, we may not be 
able to fHI your request. 
Purpoea of Form.— Use this form to 
request a copy of a tax return or tax account 
Infonnation. 

Note: If you had your return filled out by a 
paid pref>^ren check first to see if you can 
get a copy. This may save you both time and 
money, 

if you are not the taxpayer shown in item 
1, you must send a copy of your 
authorization to receive the information. 
This will generally be a power of attorney, 
tax information authorization, or evidence 
of entitlement (for Title 11 Bankruptcy or 
Receivership Proceeding). If the taxpayer is 
O MMd, you ffiust send enough evklance 
F R I r rtaMish that you are authorized to act 
he Uxpiy f *s ttete. 



Tax returns and return information about 
joint returns may be disck>sed to either the 
husband or the wife. Only one signature is 
required. If your name has changed, sign 
Form 4506 exactly as your name appeared 
on the return and also s^ with your current 
name. 

Please allcw at least 45 days for delivery 
when requesting a copy of a return, or at 
least 30 days whan requesting return 
»nformatk>n. (You must allow at least 6 
weeks processing time after a return is filed 
before requesting a copy or other 
informatkm.) 

Corporatlonit Partnerthlps, Estates, and 

Trusts.~For niles on who noay obtain tax 
informatton on the entity, see Internal 
Revenue Co6e section 6103. 
Hems 3 and 4.~lf you have named 
someone elae to receive the infomiation 
(such as a CPA, scholarship board, or 
mortgige issuer), you must Include the 
name of an Individual with the address in 
item 3. AisOi be sure and write the name of 
the client, student, or applkuint in item 4 if it 
is different from the name shown in item 1. 



MS: 



student applying for financial akj. Show the 
student's name in item 4 so the scholarship 
board will know what file to associate the 
return information with. If we cannot find a 
record of your return, we will notify the third 
party directly that we cannot fill the 
request. 

item 5.— For indivWuals, the social^ 
security number is written 000^0000. 
For businesses and certain others, the 
empk>yer Mentifk:ation number is written 
00*0000000. Please separate the nine 
digits as shown, to distinguish the type of 
number being reported. 
Ittm Enter the year(s) of the tax form 
you are requestir«. For fiscal-year filers or 
requests for quarterly returns, enter the 
date the oeriod ended. If you need mori^ 
than four different perk>ds, use additional 
request forms. Returns which were filed six 
or more years ago may not be available for 
making copies. However, tax account 
information is generally still available for 
these periods. 

(Continued on back) 
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UNITED STATES DEPARTMENT OF EDUCATION 

WASKINGTON, D.C. 20202 



ASSISTANT SECRETARY 
FOR PCSTSECONDARY EDUCATION 
OFFICE OF STUDENT FINANCIAL ASSISTANCE 



May 15, 1986 



Dear Quality Control Study Respondent: 

It has been brought to ray attention that you refused to 
participate in the Title IV Quality Control Study when an 
interviewer from Westat contacted you to arrange an appointment. 
I would like to strongly urge you to reconsider this decision. 

If you (or your son or daughter) received a Pell Grant, you 
are required to provide information and documentation to verify 
your 1) adjusted gross income, 2) U.S. taxes paid, 3) number in 
household^ 4) number in college, 5) dependency status, and 
6) untaxed income. This is in accord with the statement you 
signed on the application form. Failure to provide this 
information max result in changes in the amount of the student s 
Pell Grant. Also, if the information collected indicates that 
the student aid award is incorrect, it may be necessary to 
correct the error. You are not required to provide any other 
documentation, but would appreciate your cooperation in 
completing the remainder of the interview. The documentation you 
provide will be used to improve the delivery of student aid. 

Because you vercs randomly selected to participate in this 
study, you are representing thousands of studep- aid recipients 
(and their parents) from schools throughout the country. We 
cannot simply interview someone else. It is important that the 
views, experiences, and documentation of each sampled student and 
his or her parents ba included so that we can project the 
findings to the total recipient population nationwide. 

Again, I urge you to cooperate in our efforts to improve the 
student financial aid programs. The information you provide is 
vital to the success of the study. A Westat Supervisor will call 
you in a few days to answer any questions you might have and 
arrange for an interview. If you have a busy schedule, an 
appointment can be established at a time convenient for you. 
Thank you for your cooperation. 



Sincerely, 




Ernst Becker, Director 



Division of Quality 
Assurance 



Debt Collection and 
Management Assistance 
Service 
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UNITED STATES DEPARTMENT OF EDUCATION 



WASHINGTON. D.C. 20202 



ASSISTANT SECRETARY 
FOR POSTSECONDARY EDUCATION 
OFFICE OF STUDENT FINANCIAL ASSISTANCE 



May 15, 1986 



Dear Quality Control Study Respondent: 

As you know, Westat is conducting a quality control study of 
the Title IV student financial aid programs for the Department of 
Education. Vnen you were contacted for this study by a Westat 
interviewer, you indicated that you did not wish to participate. 
I would like to take this opportunity to explain why we're asking 
you to volunteer some of your valueible time. 

We really do need your help. As a student aid recipient (or 
parent of a recipient), you were randomly selected to participate 
in this study. Because you were selected to represent thousands 
of student aid recipients from schools throughout the country, we 
cannot simply interview someone else. It is important that the 
views, experiences, and information from each sampled student and 
his or her parents be included so that we can project the 
findings to the total recipient population nationwide. 

The decision to be interviewed or not interviewed is yours 
to make, but the help that you and other students and parents can 
give us is vital to the success of this study. The study is 
b<^ing conducted according to the regulations of the Privacy Act, 
and the information you provide will be handled according to this 
law. The authority for collecting this information is in 
Title IV of the Higher Education Act of 1965. 

Again, I would like to urge you to participate in our 
efforts to improve the student aid programs. In a few days, a 
Westat Supervisor will call you to answer any questions you might 
have and arrange for an interview. If you have a busy schedule, 
an appointment time can be established at a time convenient to 
you. Thank you for your cooperation. 



Sincerely, 




Ernst Becker, Director 



Division of Quality 
Assurance 



Debt Collection and 
Management Assistance 
Service 
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UNITED STATES DEPARTMENT OF EDUCATION 

WASHINGTON. O.C. 20202 



ASSISTANT SECRETARY 
FOR POSTSECONDARY EDUCATION 
OFFICE OF STUDENT FINANCIAL ASSISTANCE 



Jiine 2, 1986 



Dear Title IV Quality Control Study Participants 

I first want to thank you for your participation in the 
Title IV Quality Control Study that Westat is conducting for the 
U.S. Department of Education. The answers that you have provided 
to our interviewer will be of critical importance in helping the 
Department of Education improve the delivezry of Federal financial 
aid to students who need it. 

IThere is, however, an additional favor that I would like to 
ask. Because of an xinexpected delay in processing, the release 
form that you signed and dated so that we could get copies of 
your 1984 tax return is past the 60«-day period allowed by the 
Internal Revenue Service. I have enclosed the original form that 
you signed last March. Would you please sign and date the form 
^.gain above the old signature and date so that we can get copies 
of your return? X have included a stamped envelope for you to 
send the signed form back to Westat before Jxine 15, 1986. 

I want to remind you that your signature on this release is 
not mandatory. While we regard your participation as important 
to the success of this study, there is no penalty nor any affect 
on your (or your son's or daughter's) student financial aid award 
if you do not sign this form* 

Once again, thank you for your help. We sincerely appreci- 
ate the important contribution that your responses make to this 
timely research study. 



Sincerely, 




Ernst Becker . . 

Director, Division of Quality 



Assurance 
Debt Collection and Management 



Assistance Service 
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UNITED STATES DEPARTMENT OF EDUCATION 



WASHINGTON. D.C. 20202 



ASSISTANT SECRETARY 
FOR POSTSECONDARY EDUCATION 
OFFICE OF STUDENT FINANCIAL ASSISTANCE 



Dear Title IV Quality Control Study Participant: - 

Several weaks ago, we sent you a letter indicating that the 
time had expired on the form that you had signed authorizing the 
Department of Education to receive copies of your 1984 income ta: 
return in connection with our study of the Title IV student 
financial aid programs. Included with that letter was the 
original release form that you had signed, which you could simpl 
si.gn and date again. 

We have, however, not hoard from you. 

We are certainly aware of how hectic this time of year is 
for most students and their families. With that in mind, I have 
included another copy of the form in case the first one was lost 
Would you please fill this one out and send it in? 

I would like to remind you that your signature on this 
release is not mandatory. While we regard your participation as 
important to the success of this study, there is no penalty or 
any effect on your (or your son's or daughter's) student 
financial aid award if you do not sign, this form- 

Thank you again for your participation in this important 
study. Your assistance will help us to continue to improve the 
Federal student financial aid programs. 



Sincerely, 




Ernst Becker 

Director, Division of Quality 



Assurance 



Debt Collection and Management 



Assistance Service 



Enclosures 
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Advanced Technology, Inc. 

12001 Sunrise Valley Drive 
Rt'ston, Virginia 22091 
(703) 620-8000 



^20 



